Title
summary
• 76 min read
Introduction
In this Notebook, I have tried to build a model which predicts the age of a person based on his or her face.
The techniques I have used to build this Image Regression model are based on Fastai's CNN models.
The dataset I have used for this task are as follows:
- UTK Face Data
- Appa Real Face Data
- IMDB Wiki Face Data
In the course of this analysis, I have learnt many things:
- How to augment the data using Fastai's Image augmentation techinues.
- Loading and using Cadene's Pretrained Models (https://github.com/Cadene/pretrained-models.pytorch)
- How to use Image resizing technique which basically refers to gradually increase the size of image while training. This helps in achieving greater prediction accuracy.
- Discriminative Layers Training technique
# This Python 3 environment comes with many helpful analytics libraries installed
# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python
# For example, here's several helpful packages to load in
import numpy as np # linear algebra
import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)
# Input data files are available in the "../input/" directory.
# For example, running this (by clicking run or pressing Shift+Enter) will list the files in the input directory
import os
print(os.listdir("../input"))
# Any results you write to the current directory are saved as output.
import warnings
warnings.filterwarnings('ignore')
import gc
gc.collect()
['picture-2', 'picture4', 'wiki-face-data', 'picture-1', 'utk-face-cropped', 'picture-6', 'imdb-wiki-faces-dataset', 'picture-5', 'picture-3', 'appa-real-face-cropped']
0
!pip install pretrainedmodels
%reload_ext autoreload
%autoreload 2
%matplotlib inline
!pip install fastai==1.0.57
import fastai
from fastai import *
from fastai.vision import *
from torchvision.models import *
import pretrainedmodels
from utils import *
import sys
from fastai.callbacks.hooks import *
from fastai.callbacks.tracker import EarlyStoppingCallback
from fastai.callbacks.tracker import SaveModelCallback
path_wiki = Path('../input/wiki-face-data/wiki_crop/wiki_crop/')
path_imdb = Path('../input/imdb-wiki-faces-dataset/imdb_crop/imdb_crop/')
Collecting pretrainedmodels
Downloading https://files.pythonhosted.org/packages/84/0e/be6a0e58447ac16c938799d49bfb5fb7a80ac35e137547fc6cee2c08c4cf/pretrainedmodels-0.7.4.tar.gz (58kB)
|████████████████████████████████| 61kB 4.9MB/s eta 0:00:011
Requirement already satisfied: torch in /opt/conda/lib/python3.6/site-packages (from pretrainedmodels) (1.2.0)
Requirement already satisfied: torchvision in /opt/conda/lib/python3.6/site-packages (from pretrainedmodels) (0.4.0a0+6b959ee)
Requirement already satisfied: munch in /opt/conda/lib/python3.6/site-packages (from pretrainedmodels) (2.3.2)
Requirement already satisfied: tqdm in /opt/conda/lib/python3.6/site-packages (from pretrainedmodels) (4.36.1)
Requirement already satisfied: numpy in /opt/conda/lib/python3.6/site-packages (from torch->pretrainedmodels) (1.16.4)
Requirement already satisfied: six in /opt/conda/lib/python3.6/site-packages (from torchvision->pretrainedmodels) (1.12.0)
Requirement already satisfied: pillow>=4.1.1 in /opt/conda/lib/python3.6/site-packages (from torchvision->pretrainedmodels) (5.4.1)
Building wheels for collected packages: pretrainedmodels
Building wheel for pretrainedmodels (setup.py) ... done
Created wheel for pretrainedmodels: filename=pretrainedmodels-0.7.4-cp36-none-any.whl size=60963 sha256=b82608a0218b0ec36665a9794d3c75dedcac7ece90c42fab8c881ef88f2d2389
Stored in directory: /tmp/.cache/pip/wheels/69/df/63/62583c096289713f22db605aa2334de5b591d59861a02c2ecd
Successfully built pretrainedmodels
Installing collected packages: pretrainedmodels
Successfully installed pretrainedmodels-0.7.4
Requirement already satisfied: fastai==1.0.57 in /opt/conda/lib/python3.6/site-packages (1.0.57)
Requirement already satisfied: packaging in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (19.2)
Requirement already satisfied: scipy in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (1.2.1)
Requirement already satisfied: beautifulsoup4 in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (4.8.0)
Requirement already satisfied: pyyaml in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (5.1.2)
Requirement already satisfied: spacy>=2.0.18 in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (2.1.8)
Requirement already satisfied: numexpr in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (2.6.9)
Requirement already satisfied: bottleneck in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (1.2.1)
Requirement already satisfied: matplotlib in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (3.0.3)
Requirement already satisfied: typing; python_version < "3.7" in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (3.6.6)
Requirement already satisfied: nvidia-ml-py3 in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (7.352.0)
Requirement already satisfied: Pillow in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (5.4.1)
Requirement already satisfied: pandas in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (0.25.1)
Requirement already satisfied: torchvision in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (0.4.0a0+6b959ee)
Requirement already satisfied: fastprogress>=0.1.19 in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (0.1.21)
Requirement already satisfied: dataclasses; python_version < "3.7" in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (0.6)
Requirement already satisfied: torch>=1.0.0 in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (1.2.0)
Requirement already satisfied: numpy>=1.15 in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (1.16.4)
Requirement already satisfied: requests in /opt/conda/lib/python3.6/site-packages (from fastai==1.0.57) (2.22.0)
Requirement already satisfied: six in /opt/conda/lib/python3.6/site-packages (from packaging->fastai==1.0.57) (1.12.0)
Requirement already satisfied: pyparsing>=2.0.2 in /opt/conda/lib/python3.6/site-packages (from packaging->fastai==1.0.57) (2.4.2)
Requirement already satisfied: soupsieve>=1.2 in /opt/conda/lib/python3.6/site-packages (from beautifulsoup4->fastai==1.0.57) (1.9.3)
Requirement already satisfied: blis<0.3.0,>=0.2.2 in /opt/conda/lib/python3.6/site-packages (from spacy>=2.0.18->fastai==1.0.57) (0.2.4)
Requirement already satisfied: thinc<7.1.0,>=7.0.8 in /opt/conda/lib/python3.6/site-packages (from spacy>=2.0.18->fastai==1.0.57) (7.0.8)
Requirement already satisfied: plac<1.0.0,>=0.9.6 in /opt/conda/lib/python3.6/site-packages (from spacy>=2.0.18->fastai==1.0.57) (0.9.6)
Requirement already satisfied: srsly<1.1.0,>=0.0.6 in /opt/conda/lib/python3.6/site-packages (from spacy>=2.0.18->fastai==1.0.57) (0.1.0)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /opt/conda/lib/python3.6/site-packages (from spacy>=2.0.18->fastai==1.0.57) (1.0.2)
Requirement already satisfied: wasabi<1.1.0,>=0.2.0 in /opt/conda/lib/python3.6/site-packages (from spacy>=2.0.18->fastai==1.0.57) (0.2.2)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /opt/conda/lib/python3.6/site-packages (from spacy>=2.0.18->fastai==1.0.57) (2.0.2)
Requirement already satisfied: preshed<2.1.0,>=2.0.1 in /opt/conda/lib/python3.6/site-packages (from spacy>=2.0.18->fastai==1.0.57) (2.0.1)
Requirement already satisfied: cycler>=0.10 in /opt/conda/lib/python3.6/site-packages (from matplotlib->fastai==1.0.57) (0.10.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /opt/conda/lib/python3.6/site-packages (from matplotlib->fastai==1.0.57) (1.1.0)
Requirement already satisfied: python-dateutil>=2.1 in /opt/conda/lib/python3.6/site-packages (from matplotlib->fastai==1.0.57) (2.8.0)
Requirement already satisfied: pytz>=2017.2 in /opt/conda/lib/python3.6/site-packages (from pandas->fastai==1.0.57) (2019.2)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.6/site-packages (from requests->fastai==1.0.57) (2019.9.11)
Requirement already satisfied: idna<2.9,>=2.5 in /opt/conda/lib/python3.6/site-packages (from requests->fastai==1.0.57) (2.8)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/lib/python3.6/site-packages (from requests->fastai==1.0.57) (1.24.2)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /opt/conda/lib/python3.6/site-packages (from requests->fastai==1.0.57) (3.0.4)
Requirement already satisfied: tqdm<5.0.0,>=4.10.0 in /opt/conda/lib/python3.6/site-packages (from thinc<7.1.0,>=7.0.8->spacy>=2.0.18->fastai==1.0.57) (4.36.1)
Requirement already satisfied: setuptools in /opt/conda/lib/python3.6/site-packages (from kiwisolver>=1.0.1->matplotlib->fastai==1.0.57) (41.2.0)
import scipy.io
import numpy as np
import pandas as pd
from datetime import datetime, timedelta
mat_wiki = scipy.io.loadmat('../input/wiki-face-data/wiki_crop/wiki_crop/wiki.mat')
mat_imdb = scipy.io.loadmat('../input/imdb-wiki-faces-dataset/imdb_crop/imdb_crop/imdb.mat')
columns = ["dob", "photo_taken", "full_path", "gender", "name", "face_location",
"face_score", "second_face_score", 'celeb_names', 'celeb_id']
instances_wiki = mat_wiki['wiki'][0][0][0].shape[1]
instances_imdb = mat_imdb['imdb'][0][0][0].shape[1]
df_wiki = pd.DataFrame(index = range(0,instances_wiki), columns = columns)
df_imdb = pd.DataFrame(index = range(0,instances_imdb), columns = columns)
for i in mat_wiki:
if i == "wiki":
current_array = mat_wiki[i][0][0]
for j in range(len(current_array)):
#print(columns[j],": ",current_array[j])
df_wiki[columns[j]] = pd.DataFrame(current_array[j][0])
for i in mat_imdb:
if i == "imdb":
current_array = mat_imdb[i][0][0]
for j in range(len(current_array)):
#print(columns[j],": ",current_array[j])
df_imdb[columns[j]] = pd.DataFrame(current_array[j][0])
df_wiki.head()
| dob | photo_taken | full_path | gender | name | face_location | face_score | second_face_score | celeb_names | celeb_id | |
|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 723671 | 2009 | [17/10000217_1981-05-05_2009.jpg] | 1.0 | [Sami Jauhojärvi] | [[111.29109473290997, 111.29109473290997, 252.... | 4.300962 | NaN | NaN | NaN |
| 1 | 703186 | 1964 | [48/10000548_1925-04-04_1964.jpg] | 1.0 | [Dettmar Cramer] | [[252.48330229530742, 126.68165114765371, 354.... | 2.645639 | 1.949248 | NaN | NaN |
| 2 | 711677 | 2008 | [12/100012_1948-07-03_2008.jpg] | 1.0 | [Marc Okrand] | [[113.52, 169.83999999999997, 366.08, 422.4]] | 4.329329 | NaN | NaN | NaN |
| 3 | 705061 | 1961 | [65/10001965_1930-05-23_1961.jpg] | 1.0 | [Aleksandar Matanović] | [[1, 1, 634, 440]] | -inf | NaN | NaN | NaN |
| 4 | 720044 | 2012 | [16/10002116_1971-05-31_2012.jpg] | 0.0 | [Diana Damrau] | [[171.61031405173117, 75.57451239763239, 266.7... | 3.408442 | NaN | NaN | NaN |
df_imdb.head()
| dob | photo_taken | full_path | gender | name | face_location | face_score | second_face_score | celeb_names | celeb_id | |
|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 693726 | 1968 | [01/nm0000001_rm124825600_1899-5-10_1968.jpg] | 1.0 | [Fred Astaire] | [[1072.926, 161.838, 1214.7839999999999, 303.6... | 1.459693 | 1.118973 | ['Lee' George Quinones] | 6488 |
| 1 | 693726 | 1970 | [01/nm0000001_rm3343756032_1899-5-10_1970.jpg] | 1.0 | [Fred Astaire] | [[477.184, 100.352, 622.592, 245.76]] | 2.543198 | 1.852008 | ['Weird Al' Yankovic] | 6488 |
| 2 | 693726 | 1968 | [01/nm0000001_rm577153792_1899-5-10_1968.jpg] | 1.0 | [Fred Astaire] | [[114.96964308962852, 114.96964308962852, 451.... | 3.455579 | 2.985660 | [2 Chainz] | 6488 |
| 3 | 693726 | 1968 | [01/nm0000001_rm946909184_1899-5-10_1968.jpg] | 1.0 | [Fred Astaire] | [[622.8855056426588, 424.21750383700805, 844.3... | 1.872117 | NaN | [50 Cent] | 6488 |
| 4 | 693726 | 1968 | [01/nm0000001_rm980463616_1899-5-10_1968.jpg] | 1.0 | [Fred Astaire] | [[1013.8590023603723, 233.8820422075853, 1201.... | 1.158766 | NaN | [A Martinez] | 6488 |
df_wiki.shape, df_imdb.shape
((62328, 10), (460723, 10))
def datenum_to_datetime(datenum):
try:
days = datenum % 1
hours = days % 1 * 24
minutes = hours % 1 * 60
seconds = minutes % 1 * 60
exact_date = datetime.fromordinal(int(datenum)) \
+ timedelta(days=int(days)) \
+ timedelta(hours=int(hours)) \
+ timedelta(minutes=int(minutes)) \
+ timedelta(seconds=round(seconds)) \
- timedelta(days=366)
return exact_date.year
except(ValueError, TypeError, OverflowError):
return np.nan
df_wiki['date_of_birth'] = df_wiki['dob'].apply(datenum_to_datetime)
df_imdb['date_of_birth'] = df_imdb['dob'].apply(datenum_to_datetime)
df_wiki['date_of_birth'].value_counts()
df_imdb['date_of_birth'].value_counts()
1969.0 16511
1978.0 16062
1971.0 15344
1976.0 14232
1970.0 14184
...
1879.0 1
2012.0 1
1883.0 1
1888.0 1
1732.0 1
Name: date_of_birth, Length: 128, dtype: int64
df_wiki['age'] = df_wiki['photo_taken'] - df_wiki['date_of_birth']
#remove pictures does not include face
df_wiki = df_wiki[df_wiki['face_score'] != -np.inf]
#some pictures include more than one face, remove them
df_wiki = df_wiki[df_wiki['second_face_score'].isna()]
#check threshold
df_wiki = df_wiki[df_wiki['face_score'] >= 3.5]
df_wiki = df_wiki.drop(columns = ['name','face_score','second_face_score','date_of_birth','face_location'])
#some guys seem to be greater than 100. some of these are paintings. remove these old guys
df_wiki = df_wiki[df_wiki['age'] <= 100]
#some guys seem to be unborn in the data set
df_wiki = df_wiki[df_wiki['age'] > 0]
df_imdb['age'] = df_imdb['photo_taken'] - df_imdb['date_of_birth']
#remove pictures does not include face
df_imdb = df_imdb[df_imdb['face_score'] != -np.inf]
#some pictures include more than one face, remove them
df_imdb = df_imdb[df_imdb['second_face_score'].isna()]
#check threshold
df_imdb = df_imdb[df_imdb['face_score'] >= 3.5]
df_imdb = df_imdb.drop(columns = ['name','face_score','second_face_score','date_of_birth','face_location'])
#some guys seem to be greater than 100. some of these are paintings. remove these old guys
df_imdb = df_imdb[df_imdb['age'] <= 100]
#some guys seem to be unborn in the data set
df_imdb = df_imdb[df_imdb['age'] > 0]
df_wiki.head()
| dob | photo_taken | full_path | gender | celeb_names | celeb_id | age | |
|---|---|---|---|---|---|---|---|
| 0 | 723671 | 2009 | [17/10000217_1981-05-05_2009.jpg] | 1.0 | NaN | NaN | 28 |
| 2 | 711677 | 2008 | [12/100012_1948-07-03_2008.jpg] | 1.0 | NaN | NaN | 60 |
| 5 | 716189 | 2012 | [02/10002702_1960-11-09_2012.jpg] | 0.0 | NaN | NaN | 52 |
| 6 | 707745 | 1971 | [41/10003541_1937-09-27_1971.jpg] | 1.0 | NaN | NaN | 34 |
| 10 | 697114 | 1950 | [99/10004299_1908-08-19_1950.jpg] | 1.0 | NaN | NaN | 42 |
df_wiki.shape, df_imdb.shape
((17096, 7), (73683, 7))
df_wiki['age'] = df_wiki['age'].apply(lambda x: int(x))
df_imdb['age'] = df_imdb['age'].apply(lambda x: int(x))
print(type(df_wiki['age']))
df_wiki['age'].value_counts()
df_imdb['age'].value_counts()
<class 'pandas.core.series.Series'>
31 2757
32 2677
29 2578
33 2545
30 2532
...
96 1
95 1
93 1
92 1
3 1
Name: age, Length: 98, dtype: int64
df_wiki = df_wiki.drop(columns=['dob', 'photo_taken'])
df_imdb = df_imdb.drop(columns=['dob', 'photo_taken'])
df_age_wiki = df_wiki.drop(columns=['gender', 'celeb_names', 'celeb_id'])
df_age_imdb = df_imdb.drop(columns=['gender', 'celeb_names', 'celeb_id'])
df_age_wiki['age'].value_counts()
24 685
25 658
26 658
27 651
23 649
...
97 1
6 1
5 1
94 1
95 1
Name: age, Length: 97, dtype: int64
df_imdb['age'].value_counts()
31 2757
32 2677
29 2578
33 2545
30 2532
...
96 1
95 1
93 1
92 1
3 1
Name: age, Length: 98, dtype: int64
df_age_wiki['full_path'] = df_age_wiki['full_path'].str.get(0)
df_age_imdb['full_path'] = df_age_imdb['full_path'].str.get(0)
df_age_wiki.dropna(axis=0, inplace=True)
df_age_imdb.dropna(axis=0, inplace=True)
df_age_wiki.head()
| full_path | age | |
|---|---|---|
| 0 | 17/10000217_1981-05-05_2009.jpg | 28 |
| 2 | 12/100012_1948-07-03_2008.jpg | 60 |
| 5 | 02/10002702_1960-11-09_2012.jpg | 52 |
| 6 | 41/10003541_1937-09-27_1971.jpg | 34 |
| 10 | 99/10004299_1908-08-19_1950.jpg | 42 |
df_age_imdb.head()
| full_path | age | |
|---|---|---|
| 7 | 02/nm0000002_rm1363385088_1924-9-16_2004.jpg | 80 |
| 12 | 02/nm0000002_rm221957120_1924-9-16_1974.jpg | 50 |
| 14 | 02/nm0000002_rm238734336_1924-9-16_1974.jpg | 50 |
| 15 | 02/nm0000002_rm2585828096_1924-9-16_2006.jpg | 82 |
| 18 | 02/nm0000002_rm2769394176_1924-9-16_2006.jpg | 82 |
df_age_wiki['age'] = df_age_wiki['age'].apply(lambda x: int(x))
df_age_imdb['age'] = df_age_imdb['age'].apply(lambda x: int(x))
df_age_wiki.shape, df_age_imdb.shape
((17096, 2), (73683, 2))
max_age = df_age_wiki["age"].max(); print(max_age)
min_age = df_age_wiki["age"].min(); print(min_age)
100 1
max_age = df_age_imdb["age"].max(); print(max_age)
min_age = df_age_imdb["age"].min(); print(min_age)
99 1
df_age_wiki.hist()
array([[<matplotlib.axes._subplots.AxesSubplot object at 0x7fcf17fc1278>]], dtype=object)
df_age_imdb.hist()
array([[<matplotlib.axes._subplots.AxesSubplot object at 0x7fcf163a30f0>]], dtype=object)
seed = 42
# # python RNG
# import random
# random.seed(seed)
# # pytorch RNGs
# import torch
# torch.manual_seed(seed)
# torch.backends.cudnn.deterministic = True
# if torch.cuda.is_available(): torch.cuda.manual_seed_all(seed)
# numpy RNG
import numpy as np
np.random.seed(seed)
path = Path('../input/')
path_utk = Path('../input/utk-face-cropped/utkcropped/utkcropped/')
def extract_age(filename):
return float(filename.stem.split('_')[0])
tfms = get_transforms(max_rotate= 10.,max_zoom=1., max_lighting=0.20, do_flip=False,
max_warp=0., xtra_tfms=[flip_lr(), brightness(change=(0.3, 0.60), p=0.7), contrast(scale=(0.5, 2), p=0.7),
crop_pad(size=600, padding_mode='border', row_pct=0.,col_pct=0.),
rand_zoom(scale=(1.,1.5)), rand_crop(),
perspective_warp(magnitude=(-0.1,0.1)),
#jitter(magnitude=(-0.05,0.05), p=0.5),
symmetric_warp(magnitude=(-0.1,0.1)) ])
path_imdb = Path('../input/imdb-wiki-faces-dataset/imdb_crop/imdb_crop/')
path_wiki = Path('../input/wiki-face-data/wiki_crop/wiki_crop/')
data_imdb = ImageList.from_df(df_age_imdb, path_imdb, cols=['full_path'], folder ='.').split_by_rand_pct(0.2, seed=42).label_from_df(label_cls=FloatList).transform(tfms, resize_method=ResizeMethod.CROP, padding_mode='border', size=128).databunch(bs=64*2,num_workers=0).normalize(imagenet_stats)
data_wiki_small = ImageList.from_df(df_age_wiki, path_wiki, cols=['full_path'], folder ='.').split_by_rand_pct(0.2, seed=42).label_from_df(label_cls=FloatList).transform(tfms, resize_method=ResizeMethod.CROP, padding_mode='border', size=128).databunch(bs=64*2,num_workers=0).normalize(imagenet_stats)
data_wiki_big = ImageList.from_df(df_age_wiki, path_wiki, cols=['full_path'], folder ='.').split_by_rand_pct(0.2, seed=42).label_from_df(label_cls=FloatList).transform(tfms, resize_method=ResizeMethod.CROP, padding_mode='border', size=256).databunch(bs=64*2,num_workers=0).normalize(imagenet_stats)
data_utk_small = ImageList.from_folder(path_utk).split_by_rand_pct(0.2, seed=42).label_from_func(extract_age, label_cls=FloatList).transform(tfms, resize_method=ResizeMethod.CROP, padding_mode='border', size=128).databunch(bs=64*2,num_workers=0).normalize(imagenet_stats)
data_utk_big = ImageList.from_folder(path_utk).split_by_rand_pct(0.2, seed=42).label_from_func(extract_age, label_cls=FloatList).transform(tfms, resize_method=ResizeMethod.CROP, padding_mode='border', size=256).databunch(bs=64*2,num_workers=0).normalize(imagenet_stats)
df_appa = pd.read_csv('/kaggle/input/appa-real-face-cropped/labels.csv')
df_appa.head()
| file_name | real_age | |
|---|---|---|
| 0 | 000000.jpg | 4 |
| 1 | 000001.jpg | 18 |
| 2 | 000002.jpg | 80 |
| 3 | 000003.jpg | 50 |
| 4 | 000004.jpg | 17 |
df_appa.rename(columns = {"file_name":"full_path", "real_age":"age"}, inplace=True)
df_appa['age'] = df_appa['age'].apply(lambda x: int(float(x)))
path = Path('../input/appa-real-face-cropped/final_files/final_files/')
path_csv = '../input/appa-real-face-cropped/'
path_folder = '../input/appa-real-face-cropped/final_files/final_files/'
data_appa_small = ImageList.from_df(df_appa, path, cols=['full_path'], folder ='.').split_by_rand_pct(0.2, seed=42).label_from_df(label_cls=FloatList).transform(tfms, resize_method=ResizeMethod.CROP, padding_mode='border', size=128).databunch(bs=64*2,num_workers=0).normalize(imagenet_stats)
data_appa_big = ImageList.from_df(df_appa, path, cols=['full_path'], folder ='.').split_by_rand_pct(0.2, seed=42).label_from_df(label_cls=FloatList).transform(tfms, resize_method=ResizeMethod.CROP, padding_mode='border', size=256).databunch(bs=64*2,num_workers=0).normalize(imagenet_stats)
data_imdb.show_batch(4, figsize=(12,12))
data_wiki_small.show_batch(4, figsize=(12,12))
data_wiki_big.show_batch(4, figsize=(12,12))
data_utk_small.show_batch(4, figsize=(12,12))
data_utk_big.show_batch(4, figsize=(12,12))
data_appa_small.show_batch(4, figsize=(12,12))
data_appa_big.show_batch(4, figsize=(12,12))
import torch
import torch.nn as nn
import torch.nn.functional as F
from pathlib import Path
import random
opt_func = partial(optim.Adam, betas=(0.9,0.99), eps=1e-5)
def resnet(pretrained=False):
pretrained = 'imagenet' if pretrained else None
model = pretrainedmodels.resnet34(pretrained=pretrained)
return model
class L1LossFlat(nn.SmoothL1Loss):
def forward(self, input:Tensor, target:Tensor) -> Rank0Tensor:
return super().forward(input.view(-1), target.view(-1))
df_utk_small = data_utk_small.to_df()
df_utk_small.head()
| x | y | |
|---|---|---|
| 0 | 39_0_1_20170116143117964.jpg.chip.jpg | 39.0 |
| 1 | 54_0_3_20170104202315035.jpg.chip.jpg | 54.0 |
| 2 | 1_0_2_20161219154715885.jpg.chip.jpg | 1.0 |
| 3 | 30_1_2_20170116192313635.jpg.chip.jpg | 30.0 |
| 4 | 41_1_0_20170117133203334.jpg.chip.jpg | 41.0 |
df_utk_small.rename(columns = {"x":"full_path", "y":"age"}, inplace=True)
df_utk_small['age'] = df_utk_small['age'].apply(lambda x: int(float(x)))
df_utk_small.head()
| full_path | age | |
|---|---|---|
| 0 | 39_0_1_20170116143117964.jpg.chip.jpg | 39 |
| 1 | 54_0_3_20170104202315035.jpg.chip.jpg | 54 |
| 2 | 1_0_2_20161219154715885.jpg.chip.jpg | 1 |
| 3 | 30_1_2_20170116192313635.jpg.chip.jpg | 30 |
| 4 | 41_1_0_20170117133203334.jpg.chip.jpg | 41 |
df_appa.head()
| full_path | age | |
|---|---|---|
| 0 | 000000.jpg | 4 |
| 1 | 000001.jpg | 18 |
| 2 | 000002.jpg | 80 |
| 3 | 000003.jpg | 50 |
| 4 | 000004.jpg | 17 |
src_wiki = '../input/wiki-face-data/wiki_crop/wiki_crop/'
src_utk = '../input/utk-face-cropped/utkcropped/utkcropped/'
src_appa = '../input/appa-real-face-cropped/final_files/'
dest = '../working/wiki_utk/'
import os
import shutil
import pathlib
pathlib.Path(dest).mkdir(parents=True, exist_ok=True)
os.listdir(dest)
[]
for root, dirs, files in os.walk(src_wiki):
for file in files:
path_file = os.path.join(root,file)
shutil.copy2(path_file, dest)
for root, dirs, files in os.walk(src_utk):
for file in files:
path_file = os.path.join(root,file)
shutil.copy2(path_file,dest)
for root, dirs, files in os.walk(src_appa):
for file in files:
path_file = os.path.join(root,file)
shutil.copy2(path_file,dest)
df_age_wiki.head()
| full_path | age | |
|---|---|---|
| 0 | 17/10000217_1981-05-05_2009.jpg | 28 |
| 2 | 12/100012_1948-07-03_2008.jpg | 60 |
| 5 | 02/10002702_1960-11-09_2012.jpg | 52 |
| 6 | 41/10003541_1937-09-27_1971.jpg | 34 |
| 10 | 99/10004299_1908-08-19_1950.jpg | 42 |
df_age_wiki['full_path'] = df_age_wiki['full_path'].str[3:]
df_age_wiki.head()
| full_path | age | |
|---|---|---|
| 0 | 10000217_1981-05-05_2009.jpg | 28 |
| 2 | 100012_1948-07-03_2008.jpg | 60 |
| 5 | 10002702_1960-11-09_2012.jpg | 52 |
| 6 | 10003541_1937-09-27_1971.jpg | 34 |
| 10 | 10004299_1908-08-19_1950.jpg | 42 |
frames = [df_age_wiki, df_utk_small, df_appa]
df_wiki_utk_appa = pd.concat(frames)
df_wiki_utk_appa.head()
| full_path | age | |
|---|---|---|
| 0 | 10000217_1981-05-05_2009.jpg | 28 |
| 2 | 100012_1948-07-03_2008.jpg | 60 |
| 5 | 10002702_1960-11-09_2012.jpg | 52 |
| 6 | 10003541_1937-09-27_1971.jpg | 34 |
| 10 | 10004299_1908-08-19_1950.jpg | 42 |
df_wiki_utk_appa.shape
(43655, 2)
os.listdir(dest)
['29_1_1_20170112234606137.jpg.chip.jpg', '48_1_0_20170105000634011.jpg.chip.jpg', '32_0_1_20170113001322715.jpg.chip.jpg', '26_0_0_20170116192554607.jpg.chip.jpg', '36561562_1888-07-24_2004.jpg', '2338017_1960-09-21_2002.jpg', '3484443_1981-06-17_2006.jpg', '35_1_1_20170113003536772.jpg.chip.jpg', '52517_1934-10-07_1964.jpg', '46348054_1920-08-10_1942.jpg', '15_1_2_20170102234824195.jpg.chip.jpg', '903342_1972-06-28_2008.jpg', '29_0_1_20170117094745480.jpg.chip.jpg', '3762844_1945-09-25_1978.jpg', '27846664_1901-11-01_2010.jpg', '1051405_1969-11-19_2007.jpg', '249274_1955-07-01_2012.jpg', '20053572_1982-04-06_2009.jpg', '24815184_1864-08-15_2004.jpg', '001081.jpg', '12653807_1970-07-21_2011.jpg', '1_1_0_20170109193704430.jpg.chip.jpg', '26537728_1951-10-11_1978.jpg', '9952833_1990-06-05_2013.jpg', '88_0_1_20170116144533236.jpg.chip.jpg', '9632534_1957-06-09_2000.jpg', '36_1_0_20170104172916913.jpg.chip.jpg', '7796_1884-06-21_1941.jpg', '39160244_1989-03-05_2012.jpg', '50_1_0_20170117092638126.jpg.chip.jpg', '29_1_0_20170104172655682.jpg.chip.jpg', '7118448_1972-09-17_2008.jpg', '38372732_1939-06-09_1960.jpg', '35990083_1960-11-24_2001.jpg', '40_1_1_20170112231504476.jpg.chip.jpg', '27_1_2_20170116170956198.jpg.chip.jpg', '38_0_0_20170117175858084.jpg.chip.jpg', '28449525_1950-03-17_1994.jpg', '682290_1951-07-24_2009.jpg', '30079345_1993-07-20_2014.jpg', '46776424_1890-12-19_1944.jpg', '24_1_1_20170112204751139.jpg.chip.jpg', '26_0_3_20170119181310597.jpg.chip.jpg', '198400_1913-02-19_1961.jpg', '29200196_1970-01-01_2014.jpg', '25842166_1987-05-23_2011.jpg', '24576620_1956-08-20_2010.jpg', '000244.jpg', '46243758_1925-03-26_1990.jpg', '18767507_1931-05-01_2008.jpg', '99_1_0_20170120134718759.jpg.chip.jpg', '12970907_1930-10-16_1948.jpg', '001372.jpg', '378848_1945-08-03_2013.jpg', '20751442_1988-03-02_2012.jpg', '2446664_1978-10-26_2008.jpg', '8869997_1981-02-13_2012.jpg', '32504267_1954-04-17_1979.jpg', '19186371_1985-06-10_2009.jpg', '34430247_1896-12-30_1944.jpg', '27_1_1_20170113001031147.jpg.chip.jpg', '1_0_3_20161220143121101.jpg.chip.jpg', '501287_1955-05-10_2007.jpg', '65_0_0_20170111193809606.jpg.chip.jpg', '46596624_1960-02-16_1979.jpg', '14950334_1971-05-26_2006.jpg', '18418711_1904-10-16_1952.jpg', '5853885_1969-08-08_2014.jpg', '36836003_1964-12-19_1990.jpg', '853348_1931-10-26_2012.jpg', '40_0_0_20170117181229922.jpg.chip.jpg', '5_1_4_20161221201952681.jpg.chip.jpg', '4877949_1983-10-27_2013.jpg', '1662630_1971-05-01_2005.jpg', '4498970_1994-02-27_2013.jpg', '24_1_2_20170116171627787.jpg.chip.jpg', '15187291_1901-05-31_1943.jpg', '1096879_1945-09-06_2005.jpg', '39_0_3_20170104220721736.jpg.chip.jpg', '28707402_1988-12-30_2012.jpg', '1_0_0_20170109191453449.jpg.chip.jpg', '42361164_1995-12-14_2014.jpg', '34564786_1961-09-13_2007.jpg', '21_1_1_20170114032154585.jpg.chip.jpg', '28120539_1989-06-25_2009.jpg', '1274866_1939-01-06_1985.jpg', '25_1_0_20170117195346437.jpg.chip.jpg', '24353667_1924-04-01_1950.jpg', '003526.jpg', '24086813_1923-05-13_1943.jpg', '2066719_1983-06-20_2013.jpg', '32_0_1_20170116215755940.jpg.chip.jpg', '006293.jpg', '006260.jpg', '603566_1940-07-18_2012.jpg', '950281_1967-10-01_2004.jpg', '30_1_1_20170114024643760.jpg.chip.jpg', '003307.jpg', '000010.jpg', '58_0_0_20170113185223064.jpg.chip.jpg', '5340721_1945-08-19_2013.jpg', '4462806_1980-11-13_2011.jpg', '21750900_1984-06-06_2009.jpg', '1_0_2_20161219221816327.jpg.chip.jpg', '15685887_1942-09-11_1966.jpg', '9236375_1971-09-02_2010.jpg', '17_1_3_20170104221735351.jpg.chip.jpg', '15823114_1954-10-06_2006.jpg', '1573777_1943-12-31_2005.jpg', '3007277_1954-09-07_2013.jpg', '38479138_1986-04-23_2006.jpg', '006755.jpg', '1480772_1942-09-26_1990.jpg', '42244963_1948-04-19_2014.jpg', '51_0_0_20170120222842138.jpg.chip.jpg', '36422331_1982-02-28_2012.jpg', '7895818_1927-04-06_1960.jpg', '44744914_1941-02-22_2007.jpg', '796076_1969-05-16_2010.jpg', '13106750_1986-09-27_2011.jpg', '24_1_2_20170116165633882.jpg.chip.jpg', '61_1_2_20170105174916768.jpg.chip.jpg', '1567044_1969-10-24_2009.jpg', '3614080_1965-08-10_2013.jpg', '11775419_1978-07-26_2008.jpg', '58_0_3_20170119211559343.jpg.chip.jpg', '21178468_1951-12-20_1973.jpg', '004508.jpg', '006975.jpg', '40800834_1971-01-31_2010.jpg', '55_0_0_20170117185729852.jpg.chip.jpg', '37201426_1985-02-03_2014.jpg', '26788886_1962-10-14_1984.jpg', '37_0_0_20170109013120280.jpg.chip.jpg', '1_1_2_20161219201453804.jpg.chip.jpg', '39611913_1993-02-15_2014.jpg', '38379760_1990-05-14_2013.jpg', '29351931_1927-01-20_1990.jpg', '30_1_0_20170116201303368.jpg.chip.jpg', '10016074_1987-07-27_2012.jpg', '24279039_1981-04-23_2008.jpg', '30687645_1992-05-29_2011.jpg', '44285586_1992-06-19_2012.jpg', '51_0_0_20170111171747340.jpg.chip.jpg', '16534354_1989-08-21_2012.jpg', '28_0_3_20170119194803955.jpg.chip.jpg', '37128181_1992-01-13_2012.jpg', '34686067_1921-09-09_1960.jpg', '45422904_1996-03-05_2015.jpg', '37126092_1991-07-06_2014.jpg', '42520670_1940-02-11_1960.jpg', '89_0_1_20170117193440263.jpg.chip.jpg', '37_0_4_20170104000748917.jpg.chip.jpg', '43619465_1989-03-22_2014.jpg', '004828.jpg', '004922.jpg', '1624907_1927-08-07_2007.jpg', '20016339_1992-02-09_2009.jpg', '29_0_1_20170108235740510.jpg.chip.jpg', '25_1_1_20170112205103738.jpg.chip.jpg', '31_1_3_20170109134554005.jpg.chip.jpg', '779097_1977-03-06_2008.jpg', '25_0_2_20170117195218570.jpg.chip.jpg', '8027145_1978-07-25_1998.jpg', '2443611_1976-05-25_2007.jpg', '87_0_1_20170117194805270.jpg.chip.jpg', '2026045_1955-05-23_2007.jpg', '15803680_1942-12-26_2008.jpg', '1712978_1977-12-10_2008.jpg', '21644506_1985-07-05_2011.jpg', '6733489_1944-11-02_1968.jpg', '32432553_1922-02-26_1947.jpg', '000381.jpg', '3354295_1995-07-09_2008.jpg', '40890850_1949-07-20_2013.jpg', '2872703_1965-09-24_2012.jpg', '19447796_1989-07-26_2012.jpg', '10977057_1980-11-24_2008.jpg', '6145763_1960-02-19_2008.jpg', '35584490_1990-01-31_2012.jpg', '204942_1955-07-14_2011.jpg', '33947707_1995-12-29_2013.jpg', '28285893_1992-02-25_2011.jpg', '4_0_0_20170110213530901.jpg.chip.jpg', '2507966_1973-08-03_2014.jpg', '005261.jpg', '32_1_0_20170117174838232.jpg.chip.jpg', '1706281_1942-06-16_2003.jpg', '16617089_1963-03-09_2009.jpg', '39211197_1943-02-24_2006.jpg', '30501540_1960-07-29_1990.jpg', '25854312_1968-05-02_2010.jpg', '45_0_3_20170119203905073.jpg.chip.jpg', '14808073_1952-08-25_2015.jpg', '002745.jpg', '1633417_1912-02-20_1982.jpg', '31789140_1980-02-25_2000.jpg', '35_1_1_20170113005402981.jpg.chip.jpg', '2802601_1927-03-13_2002.jpg', '29462622_1935-06-10_1960.jpg', '18115101_1946-06-18_1941.jpg', '42_0_1_20170117171503046.jpg.chip.jpg', '30878743_1977-11-29_2000.jpg', '1234991_1912-03-22_1952.jpg', '006942.jpg', '19487892_1983-10-11_2015.jpg', '9427912_1969-10-23_2006.jpg', '36_0_0_20170109003634534.jpg.chip.jpg', '23617680_1929-09-28_1971.jpg', '7976847_1985-07-30_2011.jpg', '42822079_1970-11-30_2013.jpg', '79_0_0_20170111205953011.jpg.chip.jpg', '37732462_1993-07-28_2012.jpg', '58_0_0_20170117191839091.jpg.chip.jpg', '30_0_1_20170113155013364.jpg.chip.jpg', '1262985_1932-05-19_1963.jpg', '750603_1942-10-21_2012.jpg', '18600259_1982-02-28_2010.jpg', '29582036_1988-06-21_2008.jpg', '2_1_3_20161220144844310.jpg.chip.jpg', '28_1_0_20170117092254042.jpg.chip.jpg', '36967227_1897-12-02_2008.jpg', '1_0_3_20161219225759904.jpg.chip.jpg', '52_0_0_20170104204156747.jpg.chip.jpg', '46498373_1938-01-26_1959.jpg', '5197788_1959-12-31_2005.jpg', '14307202_1985-01-16_2007.jpg', '13498620_1985-06-25_2014.jpg', '1467908_1927-01-01_1948.jpg', '6777493_1985-01-09_2013.jpg', '38842697_1993-08-09_2014.jpg', '17264219_1933-04-03_1956.jpg', '7725612_1974-07-07_2011.jpg', '002618.jpg', '4_0_0_20170110213253207.jpg.chip.jpg', '30268663_1970-03-01_2013.jpg', '29797248_1989-10-22_2013.jpg', '32_1_3_20170119155703795.jpg.chip.jpg', '40519566_1941-01-10_1975.jpg', '004994.jpg', '28790298_1988-03-20_2011.jpg', '32888889_1976-12-15_2014.jpg', '13069966_1959-05-09_2006.jpg', '003129.jpg', '201160_1914-09-15_1968.jpg', '8347154_1935-12-09_1955.jpg', '28732281_1986-09-24_2010.jpg', '21346551_1931-01-09_1958.jpg', '2486630_1948-01-27_2006.jpg', '476099_1946-02-05_2009.jpg', '8006688_1961-03-14_2007.jpg', '4442797_1971-09-16_2014.jpg', '29950756_1986-02-28_2015.jpg', '16_0_0_20170110231218369.jpg.chip.jpg', '11496750_1930-05-19_1986.jpg', '44681795_1984-09-13_2015.jpg', '3591776_1946-12-31_1972.jpg', '3_1_4_20161221193144389.jpg.chip.jpg', '1984786_1930-11-23_2004.jpg', '4065567_1942-08-26_1998.jpg', '39535197_1988-11-28_2014.jpg', '1982061_1975-03-15_2012.jpg', '2593429_1947-09-19_1976.jpg', '769156_1882-02-04_1944.jpg', '45555747_1956-06-03_2015.jpg', '88833_1916-07-01_1945.jpg', '38032109_1992-04-18_2015.jpg', '002553.jpg', '7052971_1978-08-15_2010.jpg', '30760593_1946-02-05_1968.jpg', '26_0_1_20170113135516970.jpg.chip.jpg', '870163_1899-09-22_1948.jpg', '004412.jpg', '6056366_1954-11-11_1950.jpg', '28624328_1953-08-23_1973.jpg', '55_0_0_20170120221856526.jpg.chip.jpg', '1607852_1976-08-06_2007.jpg', '21_1_4_20170103223204087.jpg.chip.jpg', '10982693_1955-12-10_1992.jpg', '17958040_1987-07-24_2011.jpg', '20670994_1961-05-03_1988.jpg', '32216891_1935-05-24_1960.jpg', '67291_1937-03-22_1960.jpg', '50_0_0_20170117153725428.jpg.chip.jpg', '44_1_1_20170117142913571.jpg.chip.jpg', '004467.jpg', '24_1_0_20170117150535820.jpg.chip.jpg', '1_0_2_20161219151026723.jpg.chip.jpg', '9_0_2_20170110224822473.jpg.chip.jpg', '40_0_0_20170116221022897.jpg.chip.jpg', '35798200_1987-08-03_2014.jpg', '002576.jpg', '222737_1955-10-12_2007.jpg', '25644669_1953-10-28_2009.jpg', '45_0_0_20170116212320846.jpg.chip.jpg', '5841825_1985-03-15_2012.jpg', '60_0_0_20170120223043268.jpg.chip.jpg', '46340639_1945-04-03_1969.jpg', '17256613_1950-04-21_1984.jpg', '20050887_1973-07-21_1996.jpg', '005593.jpg', '46730387_1922-05-27_2013.jpg', '33415719_1937-03-23_1962.jpg', '24_1_0_20170117150310308.jpg.chip.jpg', '719076_1960-09-04_2012.jpg', '14146965_1983-11-27_2013.jpg', '7391254_1936-08-21_1962.jpg', '38594467_1992-03-08_2014.jpg', '32222137_1984-11-24_2010.jpg', '27_1_1_20170116214220570.jpg.chip.jpg', '10046706_1933-04-21_1962.jpg', '000005.jpg', '32_1_0_20170103180822745.jpg.chip.jpg', '30049841_1969-11-04_2015.jpg', '429097_1954-06-02_2007.jpg', '85_0_0_20170111210559755.jpg.chip.jpg', '001082.jpg', '3062692_1965-07-31_2011.jpg', '52_0_0_20170117135533022.jpg.chip.jpg', '32227332_1992-08-24_2015.jpg', '1_0_2_20161219204958037.jpg.chip.jpg', '1809170_1967-08-01_2011.jpg', '20816070_1971-05-10_2007.jpg', '35_1_0_20170103182732026.jpg.chip.jpg', '27637550_1917-03-10_2007.jpg', '15_0_0_20170104013333801.jpg.chip.jpg', '39961436_1925-01-26_1948.jpg', '21291667_1913-10-02_1950.jpg', '006552.jpg', '40492912_1988-06-30_2013.jpg', '42451669_1983-04-12_2012.jpg', '329524_1945-10-24_2014.jpg', '24_1_1_20170112192919494.jpg.chip.jpg', '8_0_0_20170110220118138.jpg.chip.jpg', '38416948_1962-07-13_1991.jpg', '33791098_1979-10-04_2011.jpg', '28_0_1_20170113142308413.jpg.chip.jpg', '40210503_1992-08-30_2014.jpg', '9509458_1993-01-31_2008.jpg', '39098401_1921-10-29_1952.jpg', '23_1_0_20170117144740562.jpg.chip.jpg', '42_1_2_20170116162349143.jpg.chip.jpg', '41978653_1951-07-27_2008.jpg', '88371_1944-05-20_2013.jpg', '44_1_3_20170119172158321.jpg.chip.jpg', '20329296_1983-10-13_2008.jpg', '001972.jpg', '31976668_1991-03-02_2015.jpg', '007118.jpg', '32752294_1973-02-07_1997.jpg', '50_1_0_20170117155139407.jpg.chip.jpg', '002627.jpg', '39220677_1976-04-22_1998.jpg', '26_0_3_20170119180948868.jpg.chip.jpg', '5101720_1975-06-30_2014.jpg', '34686543_1948-01-09_1992.jpg', '14276461_1983-04-18_2011.jpg', '5445094_1936-08-13_2012.jpg', '7_0_2_20161219193457075.jpg.chip.jpg', '1_1_2_20161219162243182.jpg.chip.jpg', '3060227_1974-10-02_2012.jpg', '39903017_1989-08-26_2013.jpg', '27962750_1894-03-20_1946.jpg', '42149298_1978-09-03_2014.jpg', '006396.jpg', '27_0_0_20170116204136393.jpg.chip.jpg', '10007577_1988-06-16_2007.jpg', '44835846_1946-12-08_1978.jpg', '26_1_1_20170116162111804.jpg.chip.jpg', '13713930_1976-06-27_2012.jpg', '27_1_2_20170116163622032.jpg.chip.jpg', '004115.jpg', '27_1_3_20170109132029072.jpg.chip.jpg', '003751.jpg', '9201851_1923-10-10_1953.jpg', '27_1_4_20170103235051403.jpg.chip.jpg', '13551558_1980-09-17_2010.jpg', '15483779_1950-05-02_1950.jpg', '1057264_1947-03-06_1974.jpg', '001200.jpg', '32_1_1_20170112212952111.jpg.chip.jpg', '41942456_1987-06-18_2014.jpg', '40_1_0_20170117160718607.jpg.chip.jpg', '3538953_1967-05-14_2013.jpg', '5928640_1957-06-14_2013.jpg', '18933035_1982-08-26_2012.jpg', '27_1_0_20170117120642103.jpg.chip.jpg', '37_0_1_20170117134417970.jpg.chip.jpg', '42_1_0_20170117121915102.jpg.chip.jpg', '3603119_1940-08-28_2011.jpg', '4163901_1950-08-20_1974.jpg', '23814332_1990-02-21_2007.jpg', '35_1_0_20170117170314803.jpg.chip.jpg', '35_0_3_20170104214246948.jpg.chip.jpg', '2415032_1954-02-19_2012.jpg', '8277549_1979-11-06_2005.jpg', '44117991_1918-02-14_1944.jpg', '42_0_0_20170104184335702.jpg.chip.jpg', '004633.jpg', '27857682_1916-11-13_1970.jpg', '14475885_1953-02-25_2010.jpg', '17_0_0_20170114025913647.jpg.chip.jpg', '36_1_1_20170113011506296.jpg.chip.jpg', '297985_1911-07-16_1955.jpg', '44400523_1928-07-26_1971.jpg', '35_1_0_20170116222743531.jpg.chip.jpg', '28_0_2_20170116191256645.jpg.chip.jpg', '40773413_1985-05-18_2013.jpg', '918149_1890-07-11_1943.jpg', '7218188_1967-05-04_2012.jpg', '81_0_1_20170117193731174.jpg.chip.jpg', '70_0_0_20170117160355895.jpg.chip.jpg', '6215810_1973-11-27_2008.jpg', '18_1_0_20170109213856511.jpg.chip.jpg', '55_0_0_20170120223706509.jpg.chip.jpg', '003723.jpg', '3569434_1980-10-01_2012.jpg', '25_1_0_20170117193539276.jpg.chip.jpg', '19216583_1937-05-10_1964.jpg', '12636288_1975-11-30_2008.jpg', '41347756_1991-03-10_2012.jpg', '12417728_1949-02-02_1980.jpg', '37684007_1945-05-07_2012.jpg', '004260.jpg', '16_1_0_20170109213608540.jpg.chip.jpg', '44695408_1911-02-24_1991.jpg', '1639687_1967-07-14_1996.jpg', '19345924_1988-09-13_2008.jpg', '2062331_1976-06-18_2013.jpg', '3628240_1960-12-17_1987.jpg', '10529976_1930-05-20_1954.jpg', '27_0_1_20170117175753874.jpg.chip.jpg', '46_0_3_20170119204802431.jpg.chip.jpg', '41470713_1952-12-26_1974.jpg', '180733_1935-05-08_1969.jpg', '18900914_1985-01-02_2006.jpg', '22421796_1955-06-25_1983.jpg', '71_1_0_20170110140328613.jpg.chip.jpg', '2476333_1981-06-01_2010.jpg', '17_1_0_20170109212920120.jpg.chip.jpg', '1167372_1903-08-14_1952.jpg', '16555375_1988-11-12_2015.jpg', '2323881_1971-03-05_2007.jpg', '2_0_2_20161219141647272.jpg.chip.jpg', '16_0_0_20170110232332618.jpg.chip.jpg', '40826007_1996-05-31_2014.jpg', '27091124_1988-03-14_2011.jpg', '41_0_1_20170117173648607.jpg.chip.jpg', '41_0_4_20170104205417148.jpg.chip.jpg', '36242663_1992-04-13_2012.jpg', '32999146_1949-07-22_2011.jpg', '28509106_1927-11-12_2009.jpg', '24994_1956-09-23_2014.jpg', '20865712_1853-09-05_2004.jpg', '26_0_1_20170117154704178.jpg.chip.jpg', '258756_1951-07-31_1971.jpg', '378171_1941-12-01_2011.jpg', '32942324_1927-12-13_1951.jpg', '5767123_1984-12-20_2015.jpg', '23167046_1987-08-01_2015.jpg', '23560198_1992-06-20_2014.jpg', '2_1_3_20161220145026015.jpg.chip.jpg', '004750.jpg', '43730875_1991-07-25_2014.jpg', '36245423_1917-12-27_1985.jpg', '39440583_1906-10-16_1944.jpg', '001140.jpg', '35414147_1993-12-24_2012.jpg', '37763795_1946-06-22_2009.jpg', '5482429_1956-05-15_1978.jpg', '11189557_1943-10-02_2011.jpg', '6786417_1934-01-29_1972.jpg', '87637_1961-09-26_2013.jpg', '1088551_1934-10-01_1961.jpg', '560012_1954-12-31_2009.jpg', '36890678_1955-04-19_1970.jpg', '29_1_2_20170116173225104.jpg.chip.jpg', '32466540_1964-11-09_2013.jpg', '43663635_1988-05-12_2012.jpg', '3329342_1982-02-15_2008.jpg', '2057979_1959-02-05_2013.jpg', '9612190_1986-03-05_2009.jpg', '537999_1974-01-14_2005.jpg', '1_1_4_20161221193744550.jpg.chip.jpg', '34_1_2_20170116193448303.jpg.chip.jpg', '001413.jpg', '31_1_1_20170116155750618.jpg.chip.jpg', '4095083_1987-04-26_2015.jpg', '6511626_1984-11-25_2011.jpg', '26_0_1_20170113134739602.jpg.chip.jpg', '2_0_0_20161219162303390.jpg.chip.jpg', '432575_1979-07-03_2013.jpg', '505670_1941-10-25_1974.jpg', '7144873_1987-04-12_2013.jpg', '18249084_1985-06-19_2012.jpg', '23864942_1981-05-31_2010.jpg', '12501_1941-12-08_2008.jpg', '696094_1963-04-18_2012.jpg', '003933.jpg', '45131582_1932-07-01_1954.jpg', '2275438_1977-09-28_2005.jpg', '35724503_1948-09-19_1966.jpg', '12878785_1989-04-30_2011.jpg', '34_0_3_20170119165846359.jpg.chip.jpg', '1266534_1975-11-03_2011.jpg', '16354477_1959-07-15_2008.jpg', '001617.jpg', '28_1_0_20170113001234883.jpg.chip.jpg', '11696343_1989-03-03_2009.jpg', '46518299_1923-05-28_1947.jpg', '961110_1959-09-14_2002.jpg', '20128467_1937-04-23_2008.jpg', '24476862_1989-05-11_2015.jpg', '5948348_1987-09-17_2007.jpg', '26_0_0_20170117172201507.jpg.chip.jpg', '30_0_0_20170119145340944.jpg.chip.jpg', '1_0_2_20161219162852759.jpg.chip.jpg', '13283895_1976-01-10_2000.jpg', '1230336_1955-09-25_2007.jpg', '5302234_1939-10-16_1971.jpg', '41142422_1986-11-13_2014.jpg', '1_0_0_20170110213012671.jpg.chip.jpg', '5376225_1974-03-31_2009.jpg', '31730021_1984-11-09_2011.jpg', '27_1_1_20170117120629605.jpg.chip.jpg', '21633828_1984-12-05_2013.jpg', '35_0_3_20170119183157029.jpg.chip.jpg', '24_1_0_20170117150848786.jpg.chip.jpg', '3970134_1987-03-17_1980.jpg', '3078676_1944-01-29_2008.jpg', '11210768_1922-02-13_1948.jpg', '22316377_1989-06-01_2012.jpg', '37_0_1_20170116214356844.jpg.chip.jpg', '1145996_1948-07-22_2007.jpg', '26765106_1986-01-08_2014.jpg', '65_0_2_20170117000746182.jpg.chip.jpg', '36753693_1936-11-26_1965.jpg', '41707776_1991-10-11_2014.jpg', '4148260_1989-05-17_2010.jpg', '35_0_0_20170117134955604.jpg.chip.jpg', '1254277_1984-04-18_2010.jpg', '22892544_1923-11-01_2009.jpg', '2058760_1915-12-02_2012.jpg', '46471365_1935-10-08_1960.jpg', '29_1_0_20170111182452813.jpg.chip.jpg', '70_1_3_20170119150943854.jpg.chip.jpg', '25_0_4_20170117151414412.jpg.chip.jpg', '21741673_1995-03-01_2009.jpg', '148878_1965-10-05_2005.jpg', '13392046_1929-06-05_2002.jpg', '28682121_1988-04-28_2011.jpg', '33305711_1987-07-12_2012.jpg', '25_1_3_20170117152019467.jpg.chip.jpg', '41388906_1951-04-03_1974.jpg', '36529968_1979-10-25_2011.jpg', '45522223_1902-05-19_1944.jpg', '15699247_1942-07-07_1978.jpg', '70_0_3_20170117191256001.jpg.chip.jpg', '171517_1939-04-21_2004.jpg', '49_1_0_20170104235723082.jpg.chip.jpg', '45_0_4_20170117202958949.jpg.chip.jpg', '36348127_1990-03-23_2012.jpg', '26_0_2_20170116181346232.jpg.chip.jpg', '43223_1941-12-31_2006.jpg', '45376382_1996-10-30_2015.jpg', '7999955_1950-11-20_2011.jpg', '8978138_1915-01-22_1975.jpg', '55_0_0_20170117191627092.jpg.chip.jpg', '10123489_1984-01-26_2009.jpg', '890_1981-06-07_2009.jpg', '22104041_1987-05-12_2011.jpg', '3964439_1987-01-21_2009.jpg', '4054196_1974-01-04_2005.jpg', '5562149_1979-03-06_2006.jpg', '600233_1961-06-27_2013.jpg', '17145298_1984-11-18_2011.jpg', '2102231_1947-04-18_2008.jpg', '31_1_0_20170117133121639.jpg.chip.jpg', '1631693_1913-10-27_2009.jpg', '670462_1984-09-07_2011.jpg', '11253737_1918-08-11_1947.jpg', '30_0_1_20170117202813031.jpg.chip.jpg', '29485869_1990-06-07_2012.jpg', '29841300_1985-04-25_1980.jpg', '6944155_1971-06-05_2012.jpg', '41162605_1990-09-09_1990.jpg', '5771217_1989-01-17_2013.jpg', '3280564_1981-11-26_2014.jpg', '34123747_1966-03-12_2011.jpg', '001552.jpg', '23_0_3_20170119164205782.jpg.chip.jpg', '37104937_1995-02-13_2013.jpg', '58_0_3_20170119195320690.jpg.chip.jpg', '2697684_1916-01-19_2011.jpg', '7705333_1984-09-29_2009.jpg', '25709268_1943-03-10_2015.jpg', '26_0_3_20170119162753492.jpg.chip.jpg', '2108919_1973-10-26_2014.jpg', '4262860_1926-11-09_2010.jpg', '1917633_1964-04-04_1973.jpg', '26_1_3_20170117144854965.jpg.chip.jpg', '2068375_1964-08-25_2014.jpg', '12_1_4_20170109214232071.jpg.chip.jpg', '41490857_1964-07-01_1986.jpg', '33701731_1955-08-16_2002.jpg', '1967356_1968-09-02_2007.jpg', '55_0_0_20170116193520434.jpg.chip.jpg', '1180042_1930-04-03_1961.jpg', '1258483_1981-05-04_2008.jpg', '27420958_1990-03-29_2015.jpg', '26531370_1963-10-06_2012.jpg', '1783180_2000-10-31_2011.jpg', '35_0_1_20170113141808954.jpg.chip.jpg', '37_1_0_20170117140111378.jpg.chip.jpg', '31305274_1957-06-02_2003.jpg', '21005625_1918-01-13_1945.jpg', '6743730_1921-03-28_1984.jpg', '8920426_1983-05-31_2011.jpg', '43853176_1989-09-12_2013.jpg', '38_1_0_20170117154128620.jpg.chip.jpg', '16_1_4_20170103214347949.jpg.chip.jpg', '88_0_2_20170112224251748.jpg.chip.jpg', '26970398_1968-01-02_2011.jpg', '29_0_0_20170116192401102.jpg.chip.jpg', '1050883_1942-03-16_2002.jpg', '3806612_1950-10-08_1966.jpg', '46725990_1923-08-01_1948.jpg', '177180_1899-11-22_1947.jpg', '26_1_3_20170119193116354.jpg.chip.jpg', '003313.jpg', '4098246_1978-05-31_2005.jpg', '006636.jpg', '5977658_1935-03-19_1964.jpg', '26457252_1980-11-08_2013.jpg', '26_0_2_20170104022626733.jpg.chip.jpg', '28_1_2_20170116162435106.jpg.chip.jpg', '45054492_1979-12-30_2014.jpg', '34904258_1977-07-10_2014.jpg', '30974716_1986-10-12_2014.jpg', '24720665_1989-04-28_2015.jpg', '38385843_1991-01-12_2013.jpg', '65_1_0_20170110143236073.jpg.chip.jpg', '23253790_1894-12-05_2008.jpg', '1134432_1993-01-26_2011.jpg', '10110600_1985-09-17_2012.jpg', '32_1_0_20170108224936101.jpg.chip.jpg', '39794722_1922-09-30_1958.jpg', '1267629_1947-10-04_2010.jpg', '34411645_1985-04-23_2011.jpg', '20459229_1973-05-11_2011.jpg', '27_1_0_20170116220503099.jpg.chip.jpg', '39277041_1949-06-29_1995.jpg', '27_0_0_20170116232646294.jpg.chip.jpg', '82471_1961-03-16_2011.jpg', '46672958_1956-04-23_1978.jpg', '13184655_1990-06-21_2010.jpg', '233512_1972-07-07_2010.jpg', '001508.jpg', '35847193_1989-09-21_2014.jpg', '9976273_1960-08-15_1988.jpg', '32384638_1990-07-12_2009.jpg', '2121079_1981-08-26_2009.jpg', '10_0_0_20170110220654150.jpg.chip.jpg', '9838372_1906-09-01_2008.jpg', '284313_1943-04-24_2008.jpg', '28910941_1959-05-01_2008.jpg', '004375.jpg', '2404184_1926-04-17_1946.jpg', '31_0_1_20170116010634833.jpg.chip.jpg', '40081785_1966-10-27_1956.jpg', '24_0_1_20170113150729383.jpg.chip.jpg', '12_1_4_20161223230033380.jpg.chip.jpg', '8_1_0_20170109202926726.jpg.chip.jpg', '13073532_1934-08-09_2012.jpg', '000668.jpg', '7300200_1949-06-10_1940.jpg', '004515.jpg', '26_1_2_20170104022829221.jpg.chip.jpg', '8171044_1984-09-03_2009.jpg', '26_0_0_20170113210127395.jpg.chip.jpg', '473342_1957-06-14_2014.jpg', '22748865_1981-02-09_2004.jpg', '12677067_1979-01-24_2010.jpg', '1_1_0_20170109191129918.jpg.chip.jpg', '4415323_1983-05-24_2013.jpg', '50_0_3_20170119205556344.jpg.chip.jpg', '34827214_1976-02-16_1970.jpg', '26_1_0_20170117173613245.jpg.chip.jpg', '31436814_1983-01-29_2012.jpg', '36475873_1988-02-09_2012.jpg', '67749_1931-11-06_1970.jpg', '50_0_0_20170117135317385.jpg.chip.jpg', '004366.jpg', '9104418_1899-02-28_1942.jpg', '006172.jpg', '31028436_1979-11-02_2013.jpg', '2491752_1971-10-25_2004.jpg', '35061796_1964-12-07_2011.jpg', '39369330_1984-01-09_2008.jpg', '1_0_4_20161221201454961.jpg.chip.jpg', '44065241_1941-10-18_1966.jpg', '6_1_0_20170103212206588.jpg.chip.jpg', '21_1_3_20170104231613281.jpg.chip.jpg', '33_1_0_20170103181050000.jpg.chip.jpg', '45491923_1919-02-04_1946.jpg', '001847.jpg', '000171.jpg', '181232_1962-12-09_2007.jpg', '250278_1965-12-30_2006.jpg', '3984297_1956-05-12_2013.jpg', '26196169_1984-03-20_2010.jpg', '25660936_1963-06-06_2014.jpg', '38613357_1921-07-10_1961.jpg', '2821311_1947-12-30_1979.jpg', '32438826_1992-05-18_2013.jpg', '15422496_1988-04-28_2009.jpg', '24597672_1987-11-19_2010.jpg', '004507.jpg', '332134_1957-10-04_2009.jpg', '45569355_1995-10-29_2014.jpg', '26_1_0_20170117003252979.jpg.chip.jpg', '31712416_1980-06-19_2000.jpg', '23055210_1960-03-17_2013.jpg', '865964_1968-01-01_2009.jpg', '7623721_1978-03-01_2010.jpg', '38896201_2000-12-26_2014.jpg', '21699302_1978-09-02_2007.jpg', '72_0_0_20170111171747670.jpg.chip.jpg', '76_0_2_20170112204914106.jpg.chip.jpg', '39481001_1990-07-25_2012.jpg', '35_0_3_20170119212039049.jpg.chip.jpg', '40753749_1951-09-15_2008.jpg', '26_1_4_20170116220414012.jpg.chip.jpg', '2850239_1940-07-26_1974.jpg', '1944523_1976-01-09_2008.jpg', '30306491_1954-10-15_2014.jpg', '54_1_3_20170109142901150.jpg.chip.jpg', '6107450_1961-05-31_2010.jpg', '14410782_1925-02-08_1976.jpg', '36273685_1984-02-22_2013.jpg', '30_0_0_20170117181257163.jpg.chip.jpg', '218081_1918-02-01_1960.jpg', '7088375_1980-12-19_2006.jpg', '000679.jpg', '43399688_1994-08-03_2015.jpg', '3555882_1899-05-20_1940.jpg', '41365854_1980-06-19_2004.jpg', '16536131_1949-11-09_1980.jpg', '640889_1969-09-09_2008.jpg', '163892_1958-10-20_2012.jpg', '18888856_1989-03-19_2012.jpg', '31248425_1961-11-26_1990.jpg', '001744.jpg', '25918808_1982-06-22_2012.jpg', '529315_1906-01-14_1960.jpg', '28586654_1967-05-06_2007.jpg', '5468856_1976-04-07_2009.jpg', '26_1_0_20170117175317465.jpg.chip.jpg', '26_1_1_20170116231924481.jpg.chip.jpg', '34666103_1974-04-11_1996.jpg', '23_1_4_20170103233619053.jpg.chip.jpg', '26_1_3_20170119193148570.jpg.chip.jpg', '552779_1971-11-03_2006.jpg', '25_1_0_20170103180657288.jpg.chip.jpg', '161577_1980-06-08_2011.jpg', '26_1_1_20170116160509916.jpg.chip.jpg', '004715.jpg', '1925799_1967-01-30_2010.jpg', '207774_1925-06-09_1966.jpg', '23890218_1982-03-04_2009.jpg', '3313310_1965-04-28_2009.jpg', '4_1_3_20161219224933671.jpg.chip.jpg', '24179114_1938-10-20_2008.jpg', '571810_1956-09-17_2000.jpg', '651347_1955-04-06_2015.jpg', '005314.jpg', '15552802_1961-06-09_2009.jpg', '44108810_1926-06-28_1947.jpg', '34957545_1928-03-20_2013.jpg', '23877707_1926-07-23_1945.jpg', '327664_1951-01-12_1994.jpg', '36_0_0_20170113184113374.jpg.chip.jpg', '56_1_0_20170113182536919.jpg.chip.jpg', '8808513_1960-01-10_1983.jpg', '31621448_1991-04-04_2011.jpg', '3149083_1961-03-01_1990.jpg', '731994_1973-03-19_2007.jpg', '647373_1956-07-15_2010.jpg', '45396098_1976-01-10_2015.jpg', '1643259_1923-10-13_1961.jpg', '52_0_0_20170111201135454.jpg.chip.jpg', '195152_1931-11-26_2003.jpg', '23381415_1986-11-14_2011.jpg', '44119711_1819-11-17_2004.jpg', '252442_1896-08-15_1947.jpg', '1_0_4_20161219195147803.jpg.chip.jpg', '2377004_1972-01-30_2005.jpg', '32_1_0_20170116191618394.jpg.chip.jpg', '8087946_1944-04-21_1966.jpg', '13550616_1977-12-19_2010.jpg', '9322645_1971-08-26_2007.jpg', '10503610_1949-08-01_2005.jpg', '2385757_1961-05-08_2003.jpg', '2861958_1967-03-30_2011.jpg', '33983_1951-12-20_2000.jpg', '28_0_3_20170119194929915.jpg.chip.jpg', '524636_1961-04-08_2012.jpg', '30_1_0_20170117103601281.jpg.chip.jpg', '34876346_1946-05-13_2010.jpg', '34363325_1974-01-20_2001.jpg', '80_0_0_20170117175301125.jpg.chip.jpg', '28_0_0_20170116174356539.jpg.chip.jpg', '2_0_2_20161219202702788.jpg.chip.jpg', '22522134_1977-06-22_2002.jpg', '959464_1977-04-05_2004.jpg', '20597806_1980-10-04_2012.jpg', '28_0_0_20170116212102377.jpg.chip.jpg', '2134356_1966-03-14_2007.jpg', '36_0_3_20170119195506419.jpg.chip.jpg', '26_1_0_20170116164040161.jpg.chip.jpg', '5_0_4_20170103210509634.jpg.chip.jpg', '45681223_1993-06-25_2014.jpg', '43479903_1988-07-21_2014.jpg', '67030_1909-06-20_1940.jpg', '2857671_1920-02-14_1947.jpg', '36787370_1977-04-05_2010.jpg', '12167720_1984-09-30_2014.jpg', '10970703_1984-05-21_2011.jpg', '35022161_1984-11-13_2011.jpg', '45711961_1986-08-14_2008.jpg', '47_0_0_20170103183442514.jpg.chip.jpg', '3250661_1986-01-31_2009.jpg', '1896252_1942-09-02_2008.jpg', '5737964_1980-07-16_2007.jpg', '45277231_1926-02-26_1973.jpg', '40380141_1995-11-27_2013.jpg', '343294_1964-01-27_2001.jpg', '12754_1899-07-07_1946.jpg', '25648564_1936-12-07_1971.jpg', '7463952_1982-12-30_2013.jpg', '471630_1898-11-22_1978.jpg', '1148639_1963-12-26_2009.jpg', '40184108_1993-05-29_2013.jpg', '92331_1955-08-10_2008.jpg', '10016225_1988-02-03_2012.jpg', '40_0_2_20170116191354067.jpg.chip.jpg', '11700461_1983-10-11_2007.jpg', '32908796_1985-06-20_2013.jpg', '24_1_0_20170117191400699.jpg.chip.jpg', '24064016_1989-05-07_2008.jpg', '25211052_1991-07-08_2015.jpg', '23315533_1988-01-27_2011.jpg', '33999923_1993-04-30_2012.jpg', '79_1_0_20170120225803298.jpg.chip.jpg', '25615152_1987-11-11_2009.jpg', '11973758_1991-02-27_2008.jpg', '1246851_1974-08-09_2014.jpg', '26_0_2_20170116182347867.jpg.chip.jpg', '1235669_1976-09-11_2007.jpg', '2693444_1972-08-09_1965.jpg', '27_0_4_20170103235757172.jpg.chip.jpg', '30387921_1975-02-01_2010.jpg', '3331742_1969-03-26_2013.jpg', '17021305_1883-02-28_1967.jpg', '41691585_1995-09-25_2015.jpg', '3101416_1973-03-17_2009.jpg', '53_1_0_20170109220736654.jpg.chip.jpg', '34027685_1972-07-22_2005.jpg', '7514390_1977-05-25_2014.jpg', '3348502_1984-09-18_2007.jpg', '11021240_1887-08-31_1954.jpg', '1090974_1971-03-12_1988.jpg', '35_1_1_20170113012807626.jpg.chip.jpg', '913724_1979-04-03_2008.jpg', '34557038_1974-07-08_2011.jpg', '006995.jpg', '37_1_1_20170116232758932.jpg.chip.jpg', '5929080_1943-07-02_1997.jpg', '6171937_1981-05-29_2007.jpg', '2251257_1931-03-23_1964.jpg', '2776174_1969-02-20_2014.jpg', '65_1_0_20170103184138483.jpg.chip.jpg', '7856897_1986-05-23_2012.jpg', '46182062_1969-04-29_2013.jpg', '38128011_1979-08-14_2002.jpg', '1776092_1959-01-31_2007.jpg', '4647550_1949-10-31_2012.jpg', '2092862_1956-04-09_1984.jpg', '2201_1922-06-19_1955.jpg', '68_0_0_20170117172938504.jpg.chip.jpg', '33369667_1989-01-28_2012.jpg', '8118857_1953-06-11_1977.jpg', '28968145_1985-04-03_2007.jpg', '1768989_1953-07-21_2008.jpg', '849454_1955-04-17_2013.jpg', '26908645_1973-03-22_2015.jpg', '18109220_1987-08-04_2015.jpg', '8286385_1922-11-01_1953.jpg', '1405558_1968-03-14_2012.jpg', '26_1_1_20170112212042603.jpg.chip.jpg', '21222403_1983-09-29_2012.jpg', '593572_1960-09-04_2007.jpg', '32_1_1_20170116164948959.jpg.chip.jpg', '3776811_1938-01-05_1990.jpg', '42737444_1674-04-22_2014.jpg', '18_0_0_20170117140126568.jpg.chip.jpg', '20681612_1925-11-04_1947.jpg', '30_0_2_20170117141818577.jpg.chip.jpg', '004472.jpg', '22_1_1_20170112233701320.jpg.chip.jpg', '4334396_1972-09-07_1993.jpg', '27599355_1985-04-07_2007.jpg', '60_0_0_20170117160634375.jpg.chip.jpg', '6042299_1975-01-05_1970.jpg', '3214260_1968-09-12_2012.jpg', '15208001_1981-02-08_2012.jpg', '27_0_0_20170117175924156.jpg.chip.jpg', '23542494_1976-03-21_2012.jpg', '37867537_1965-02-11_1984.jpg', '37484443_1991-03-17_2013.jpg', '1_1_3_20161219224457585.jpg.chip.jpg', '18284846_1989-09-02_2010.jpg', '2456295_1969-10-14_2008.jpg', '20_1_0_20170103201733831.jpg.chip.jpg', '41334185_1991-06-24_2014.jpg', '17814123_1930-08-16_1974.jpg', '36873309_1959-12-01_2010.jpg', '65_1_0_20170110141141881.jpg.chip.jpg', '8484508_1981-12-31_2010.jpg', '26_0_2_20170119184102087.jpg.chip.jpg', '54_0_0_20170105163714131.jpg.chip.jpg', '31_0_1_20170113175408636.jpg.chip.jpg', '18676025_1969-06-11_2008.jpg', '26_1_4_20170117154137164.jpg.chip.jpg', '001075.jpg', '28_0_0_20170119194754322.jpg.chip.jpg', '73_0_0_20170120230233019.jpg.chip.jpg', '42412485_1931-09-03_1966.jpg', '26_1_4_20170103230455729.jpg.chip.jpg', '14013061_1977-11-07_2006.jpg', '72_0_2_20170112224532491.jpg.chip.jpg', '36_0_3_20170119194656666.jpg.chip.jpg', '1_0_2_20161219201442357.jpg.chip.jpg', '7285573_1929-05-07_1962.jpg', '001164.jpg', '6015495_1958-09-27_1950.jpg', '14996024_1982-09-16_2013.jpg', '2195339_1953-11-23_2008.jpg', '38480304_1911-04-12_2011.jpg', '42219298_1990-02-14_2014.jpg', '60_0_3_20170119205525232.jpg.chip.jpg', '2161453_1950-02-03_1973.jpg', '37464471_1957-05-22_1980.jpg', '38397939_1937-06-03_1960.jpg', '33319744_1992-04-26_1990.jpg', '3428686_1965-06-18_2015.jpg', '38_0_1_20170116003732637.jpg.chip.jpg', '2178459_1985-01-14_2008.jpg', '13_1_0_20170109204325267.jpg.chip.jpg', '20545506_1976-07-08_2009.jpg', '70_0_0_20170117173727535.jpg.chip.jpg', '005296.jpg', '26_1_2_20170116175903368.jpg.chip.jpg', '26_1_4_20170117201351845.jpg.chip.jpg', '24232963_1987-10-02_2012.jpg', '4_1_1_20170117202024231.jpg.chip.jpg', '5388344_1978-06-22_2012.jpg', '1497643_1916-08-14_1954.jpg', '004802.jpg', '37791562_1991-02-15_2014.jpg', '24_1_3_20170104223019607.jpg.chip.jpg', '11787967_1967-04-02_1960.jpg', '10_0_0_20170110225444491.jpg.chip.jpg', '6731123_1953-07-05_2011.jpg', '26897013_1991-06-25_2010.jpg', '002183.jpg', '42345_1900-01-27_1955.jpg', '863211_1930-06-01_1971.jpg', '22698827_1985-12-16_2013.jpg', '10838173_1990-02-01_2012.jpg', '8224016_1973-03-15_1970.jpg', '3660144_1972-04-13_1970.jpg', '42842300_1994-10-18_2014.jpg', '1515927_1980-09-26_2005.jpg', '307526_1964-09-08_2013.jpg', '6407782_1977-12-20_2007.jpg', '35937539_1992-08-08_2012.jpg', '33124994_1992-01-07_2011.jpg', '53_1_3_20170109141903857.jpg.chip.jpg', '40_0_2_20170116181214637.jpg.chip.jpg', '23_1_0_20170117144945297.jpg.chip.jpg', '003552.jpg', '23_1_3_20170117144858514.jpg.chip.jpg', '44059509_1928-01-08_1954.jpg', '24310807_1945-01-18_2009.jpg', '38634484_1993-07-30_2012.jpg', '8980506_1979-07-14_2009.jpg', '1042604_1956-06-04_2010.jpg', '64_0_0_20170104185632214.jpg.chip.jpg', '32_1_2_20170116191623170.jpg.chip.jpg', ...]
df_wiki_utk_appa.hist()
array([[<matplotlib.axes._subplots.AxesSubplot object at 0x7fcf163f9c88>]], dtype=object)
df_wiki_utk_appa['age'].value_counts()
26 2632
24 1593
28 1575
25 1573
30 1382
...
98 2
97 2
101 1
111 1
103 1
Name: age, Length: 107, dtype: int64
df_wiki_utk_appa = df_wiki_utk_appa[df_wiki_utk_appa['age'] <= 100]
df_wiki_utk_appa = df_wiki_utk_appa[df_wiki_utk_appa['age'] > 0]
df_wiki_utk_appa['age'].min(), df_wiki_utk_appa['age'].max()
(1, 100)
df_wiki_utk_appa['age'] = df_wiki_utk_appa['age'].astype(int)
path_wiki_utk_appa = Path('../working/wiki_utk/')
np.random.seed(42)
data_wiki_small_src = (ImageList.from_df(df_wiki_utk_appa, path_wiki_utk_appa, cols=['full_path'], folder='.')
.split_by_rand_pct(0.2, seed=42)
.label_from_df(label_cls=FloatList))
# data_wiki_big = ImageList.from_df(df_wiki_utk_appa, path_wiki_utk_appa, cols=['full_path'], folder='.').split_by_rand_pct(0.2, seed=42).label_from_df(label_cls=FloatList).transform(tfms, resize_method=ResizeMethod.CROP, padding_mode='border', size=256).databunch(bs=64*2,num_workers=0).normalize(imagenet_stats)
data_wiki_small = (data_wiki_small_src.transform(tfms, resize_method=ResizeMethod.CROP, padding_mode='border', size=128)
.databunch(bs=64,num_workers=0).normalize(imagenet_stats))
data_wiki_small.show_batch(4, figsize=(12,12))
class AgeModel(nn.Module):
def __init__(self):
super().__init__()
layers = list(models.resnet34(pretrained=True).children())[:-2]
layers += [AdaptiveConcatPool2d(), Flatten()]
#layers += [nn.BatchNorm1d(4096, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)]
#layers += [nn.Dropout(p=0.60)]
#layers += [nn.Linear(4096, 1024, bias=True), nn.ReLU(inplace=True)]
#layers += [nn.BatchNorm1d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)]
#layers += [nn.Dropout(p=0.60)]
#layers += [nn.Linear(2048, 1024, bias=True), nn.ReLU(inplace=True)]
#layers += [nn.BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)]
#layers += [nn.Dropout(p=0.75)]
#layers += [nn.Linear(1024, 256, bias=True), nn.ReLU(inplace=True)]
#layers += [nn.BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)]
#layers += [nn.Dropout(p=0.50)]
#layers += [nn.Linear(512,256 , bias=True), nn.ReLU(inplace=True)]
layers += [nn.BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)]
layers += [nn.Dropout(p=0.50)]
layers += [nn.Linear(1024, 512, bias=True), nn.ReLU(inplace=True)]
layers += [nn.BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)]
layers += [nn.Dropout(p=0.50)]
layers += [nn.Linear(512, 16, bias=True), nn.ReLU(inplace=True)]
layers += [nn.Linear(16,1)]
self.agemodel = nn.Sequential(*layers)
def forward(self, x):
return self.agemodel(x).squeeze(-1)
# could add 116*torch.sigmoid
model = AgeModel()
Downloading: "https://download.pytorch.org/models/resnet34-333f7ec4.pth" to /tmp/.cache/torch/checkpoints/resnet34-333f7ec4.pth 100%|██████████| 83.3M/83.3M [00:00<00:00, 92.9MB/s]
learn = Learner(data_wiki_small, model, model_dir = "/temp/model/", opt_func=opt_func, bn_wd=False, metrics=root_mean_squared_error,
callback_fns=[ShowGraph]).mixup(stack_y=False, alpha=0.2)
learn.loss_func = L1LossFlat()
learn.split([model.agemodel[4],model.agemodel[6],model.agemodel[8]])
Learner(data=ImageDataBunch;
Train: LabelList (34909 items)
x: ImageList
Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128)
y: FloatList
52.0,34.0,36.0,23.0,78.0
Path: ../working/wiki_utk;
Valid: LabelList (8727 items)
x: ImageList
Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128)
y: FloatList
29.0,58.0,22.0,12.0,32.0
Path: ../working/wiki_utk;
Test: None, model=AgeModel(
(agemodel): Sequential(
(0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
(1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
(4): Sequential(
(0): BasicBlock(
(conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(1): BasicBlock(
(conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(2): BasicBlock(
(conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(5): Sequential(
(0): BasicBlock(
(conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(downsample): Sequential(
(0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(1): BasicBlock(
(conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(2): BasicBlock(
(conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(3): BasicBlock(
(conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(6): Sequential(
(0): BasicBlock(
(conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(downsample): Sequential(
(0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(1): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(2): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(3): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(4): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(5): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(7): Sequential(
(0): BasicBlock(
(conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(downsample): Sequential(
(0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(1): BasicBlock(
(conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(2): BasicBlock(
(conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(8): AdaptiveConcatPool2d(
(ap): AdaptiveAvgPool2d(output_size=1)
(mp): AdaptiveMaxPool2d(output_size=1)
)
(9): Flatten()
(10): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(11): Dropout(p=0.5, inplace=False)
(12): Linear(in_features=1024, out_features=512, bias=True)
(13): ReLU(inplace=True)
(14): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(15): Dropout(p=0.5, inplace=False)
(16): Linear(in_features=512, out_features=16, bias=True)
(17): ReLU(inplace=True)
(18): Linear(in_features=16, out_features=1, bias=True)
)
), opt_func=functools.partial(<class 'torch.optim.adam.Adam'>, betas=(0.9, 0.99), eps=1e-05), loss_func=L1LossFlat(), metrics=[<function root_mean_squared_error at 0x7fcf62a9bf28>], true_wd=True, bn_wd=False, wd=0.01, train_bn=True, path=PosixPath('../working/wiki_utk'), model_dir='/temp/model/', callback_fns=[functools.partial(<class 'fastai.basic_train.Recorder'>, add_time=True, silent=False), <class 'fastai.train.ShowGraph'>, functools.partial(<class 'fastai.callbacks.mixup.MixUpCallback'>, alpha=0.2, stack_x=False, stack_y=False)], callbacks=[], layer_groups=[Sequential(
(0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
(1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
), Sequential(
(0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(6): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(7): ReLU(inplace=True)
(8): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(9): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(10): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(11): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(12): ReLU(inplace=True)
(13): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(14): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(15): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(16): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(17): ReLU(inplace=True)
(18): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(19): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(20): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
(21): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(22): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(23): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(24): ReLU(inplace=True)
(25): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(26): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(27): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(28): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(29): ReLU(inplace=True)
(30): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(31): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(32): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(33): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(34): ReLU(inplace=True)
(35): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(36): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
(0): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
(6): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(7): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(8): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(9): ReLU(inplace=True)
(10): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(11): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(13): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(14): ReLU(inplace=True)
(15): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(16): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(17): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(18): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(19): ReLU(inplace=True)
(20): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(21): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(22): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(23): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(24): ReLU(inplace=True)
(25): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(26): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(27): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(28): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(29): ReLU(inplace=True)
(30): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(31): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(32): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(33): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(34): ReLU(inplace=True)
(35): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(36): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(37): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
(38): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(39): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(40): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(41): ReLU(inplace=True)
(42): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(43): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(44): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(45): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(46): ReLU(inplace=True)
(47): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(48): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
(0): AdaptiveAvgPool2d(output_size=1)
(1): AdaptiveMaxPool2d(output_size=1)
(2): Flatten()
(3): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(4): Dropout(p=0.5, inplace=False)
(5): Linear(in_features=1024, out_features=512, bias=True)
(6): ReLU(inplace=True)
(7): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(8): Dropout(p=0.5, inplace=False)
(9): Linear(in_features=512, out_features=16, bias=True)
(10): ReLU(inplace=True)
(11): Linear(in_features=16, out_features=1, bias=True)
)], add_time=True, silent=False, cb_fns_registered=False)
#learn = cnn_learner(data, resnet50, pretrained = True, model_dir = "/temp/model/",callback_fns=[ShowGraph])
#learn.loss_func = L1LossFlat()
import fastai
learn.freeze_to(-1)
learn.lr_find()
learn.recorder.plot(suggestion = True)
LR Finder is complete, type {learner_name}.recorder.plot() to see the graph.
Min numerical gradient: 3.98E-02
Min loss divided by 10: 9.12E-02
lr = 2e-2
learn.fit_one_cycle(5, max_lr=slice(lr), wd=(1e-6, 1e-4, 1e-2, 1e-1), pct_start=0.5, callbacks=[SaveModelCallback(learn)])
| epoch | train_loss | valid_loss | root_mean_squared_error | time |
|---|---|---|---|---|
| 0 | 9.605057 | 9.252867 | 13.258863 | 07:27 |
| 1 | 8.392913 | 7.800601 | 11.307045 | 07:26 |
| 2 | 7.934024 | 8.536113 | 12.432997 | 07:25 |
| 3 | 7.214382 | 6.707546 | 9.728629 | 07:25 |
| 4 | 7.115435 | 6.620564 | 9.895959 | 07:26 |
Better model found at epoch 0 with valid_loss value: 9.252866744995117. Better model found at epoch 1 with valid_loss value: 7.800601005554199. Better model found at epoch 3 with valid_loss value: 6.707545757293701. Better model found at epoch 4 with valid_loss value: 6.620563507080078.
learn.save('first_head_resnet34')
learn.load('first_head_resnet34')
Learner(data=ImageDataBunch;
Train: LabelList (34909 items)
x: ImageList
Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128)
y: FloatList
52.0,34.0,36.0,23.0,78.0
Path: ../working/wiki_utk;
Valid: LabelList (8727 items)
x: ImageList
Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128)
y: FloatList
29.0,58.0,22.0,12.0,32.0
Path: ../working/wiki_utk;
Test: None, model=AgeModel(
(agemodel): Sequential(
(0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
(1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
(4): Sequential(
(0): BasicBlock(
(conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(1): BasicBlock(
(conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(2): BasicBlock(
(conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(5): Sequential(
(0): BasicBlock(
(conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(downsample): Sequential(
(0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(1): BasicBlock(
(conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(2): BasicBlock(
(conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(3): BasicBlock(
(conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(6): Sequential(
(0): BasicBlock(
(conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(downsample): Sequential(
(0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(1): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(2): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(3): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(4): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(5): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(7): Sequential(
(0): BasicBlock(
(conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(downsample): Sequential(
(0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(1): BasicBlock(
(conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(2): BasicBlock(
(conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(8): AdaptiveConcatPool2d(
(ap): AdaptiveAvgPool2d(output_size=1)
(mp): AdaptiveMaxPool2d(output_size=1)
)
(9): Flatten()
(10): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(11): Dropout(p=0.5, inplace=False)
(12): Linear(in_features=1024, out_features=512, bias=True)
(13): ReLU(inplace=True)
(14): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(15): Dropout(p=0.5, inplace=False)
(16): Linear(in_features=512, out_features=16, bias=True)
(17): ReLU(inplace=True)
(18): Linear(in_features=16, out_features=1, bias=True)
)
), opt_func=functools.partial(<class 'torch.optim.adam.Adam'>, betas=(0.9, 0.99), eps=1e-05), loss_func=L1LossFlat(), metrics=[<function root_mean_squared_error at 0x7fcf62a9bf28>], true_wd=True, bn_wd=False, wd=0.01, train_bn=True, path=PosixPath('../working/wiki_utk'), model_dir='/temp/model/', callback_fns=[functools.partial(<class 'fastai.basic_train.Recorder'>, add_time=True, silent=False), <class 'fastai.train.ShowGraph'>, functools.partial(<class 'fastai.callbacks.mixup.MixUpCallback'>, alpha=0.2, stack_x=False, stack_y=False)], callbacks=[], layer_groups=[Sequential(
(0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
(1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
), Sequential(
(0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(6): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(7): ReLU(inplace=True)
(8): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(9): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(10): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(11): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(12): ReLU(inplace=True)
(13): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(14): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(15): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(16): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(17): ReLU(inplace=True)
(18): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(19): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(20): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
(21): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(22): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(23): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(24): ReLU(inplace=True)
(25): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(26): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(27): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(28): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(29): ReLU(inplace=True)
(30): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(31): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(32): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(33): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(34): ReLU(inplace=True)
(35): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(36): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
(0): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
(6): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(7): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(8): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(9): ReLU(inplace=True)
(10): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(11): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(13): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(14): ReLU(inplace=True)
(15): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(16): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(17): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(18): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(19): ReLU(inplace=True)
(20): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(21): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(22): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(23): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(24): ReLU(inplace=True)
(25): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(26): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(27): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(28): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(29): ReLU(inplace=True)
(30): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(31): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(32): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(33): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(34): ReLU(inplace=True)
(35): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(36): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(37): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
(38): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(39): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(40): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(41): ReLU(inplace=True)
(42): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(43): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(44): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(45): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(46): ReLU(inplace=True)
(47): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(48): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
(0): AdaptiveAvgPool2d(output_size=1)
(1): AdaptiveMaxPool2d(output_size=1)
(2): Flatten()
(3): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(4): Dropout(p=0.5, inplace=False)
(5): Linear(in_features=1024, out_features=512, bias=True)
(6): ReLU(inplace=True)
(7): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(8): Dropout(p=0.5, inplace=False)
(9): Linear(in_features=512, out_features=16, bias=True)
(10): ReLU(inplace=True)
(11): Linear(in_features=16, out_features=1, bias=True)
)], add_time=True, silent=False, cb_fns_registered=True)
learn.unfreeze()
learn.lr_find()
learn.recorder.plot(suggestion = True)
LR Finder is complete, type {learner_name}.recorder.plot() to see the graph.
Min numerical gradient: 1.91E-06
Min loss divided by 10: 6.92E-07
learn.fit_one_cycle(5, max_lr=slice(1e-6, lr/5), wd=(1e-6, 1e-4, 1e-2, 1e-1),
callbacks=[SaveModelCallback(learn)], pct_start=0.5)
| epoch | train_loss | valid_loss | root_mean_squared_error | time |
|---|---|---|---|---|
| 0 | 7.085314 | 7.008016 | 10.614280 | 07:32 |
| 1 | 7.218247 | 6.842098 | 9.879888 | 07:31 |
| 2 | 6.800269 | 6.805061 | 9.620695 | 07:30 |
| 3 | 6.229340 | 5.717019 | 8.594589 | 07:30 |
| 4 | 6.011308 | 5.505085 | 8.282001 | 07:31 |
Better model found at epoch 0 with valid_loss value: 7.008016109466553. Better model found at epoch 1 with valid_loss value: 6.842098236083984. Better model found at epoch 2 with valid_loss value: 6.805061340332031. Better model found at epoch 3 with valid_loss value: 5.7170186042785645. Better model found at epoch 4 with valid_loss value: 5.505085468292236.
x,y = next(iter(learn.data.train_dl))
x.shape
torch.Size([64, 3, 128, 128])
learn.save('first_body_resnet34')
learn.load('first_body_resnet34')
Learner(data=ImageDataBunch;
Train: LabelList (34909 items)
x: ImageList
Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128)
y: FloatList
52.0,34.0,36.0,23.0,78.0
Path: ../working/wiki_utk;
Valid: LabelList (8727 items)
x: ImageList
Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128),Image (3, 128, 128)
y: FloatList
29.0,58.0,22.0,12.0,32.0
Path: ../working/wiki_utk;
Test: None, model=AgeModel(
(agemodel): Sequential(
(0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
(1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
(4): Sequential(
(0): BasicBlock(
(conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(1): BasicBlock(
(conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(2): BasicBlock(
(conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(5): Sequential(
(0): BasicBlock(
(conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(downsample): Sequential(
(0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(1): BasicBlock(
(conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(2): BasicBlock(
(conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(3): BasicBlock(
(conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(6): Sequential(
(0): BasicBlock(
(conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(downsample): Sequential(
(0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(1): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(2): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(3): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(4): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(5): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(7): Sequential(
(0): BasicBlock(
(conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(downsample): Sequential(
(0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(1): BasicBlock(
(conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(2): BasicBlock(
(conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(8): AdaptiveConcatPool2d(
(ap): AdaptiveAvgPool2d(output_size=1)
(mp): AdaptiveMaxPool2d(output_size=1)
)
(9): Flatten()
(10): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(11): Dropout(p=0.5, inplace=False)
(12): Linear(in_features=1024, out_features=512, bias=True)
(13): ReLU(inplace=True)
(14): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(15): Dropout(p=0.5, inplace=False)
(16): Linear(in_features=512, out_features=16, bias=True)
(17): ReLU(inplace=True)
(18): Linear(in_features=16, out_features=1, bias=True)
)
), opt_func=functools.partial(<class 'torch.optim.adam.Adam'>, betas=(0.9, 0.99), eps=1e-05), loss_func=L1LossFlat(), metrics=[<function root_mean_squared_error at 0x7fcf62a9bf28>], true_wd=True, bn_wd=False, wd=0.01, train_bn=True, path=PosixPath('../working/wiki_utk'), model_dir='/temp/model/', callback_fns=[functools.partial(<class 'fastai.basic_train.Recorder'>, add_time=True, silent=False), <class 'fastai.train.ShowGraph'>, functools.partial(<class 'fastai.callbacks.mixup.MixUpCallback'>, alpha=0.2, stack_x=False, stack_y=False)], callbacks=[], layer_groups=[Sequential(
(0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
(1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
), Sequential(
(0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(6): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(7): ReLU(inplace=True)
(8): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(9): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(10): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(11): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(12): ReLU(inplace=True)
(13): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(14): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(15): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(16): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(17): ReLU(inplace=True)
(18): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(19): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(20): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
(21): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(22): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(23): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(24): ReLU(inplace=True)
(25): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(26): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(27): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(28): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(29): ReLU(inplace=True)
(30): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(31): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(32): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(33): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(34): ReLU(inplace=True)
(35): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(36): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
(0): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
(6): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(7): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(8): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(9): ReLU(inplace=True)
(10): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(11): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(13): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(14): ReLU(inplace=True)
(15): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(16): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(17): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(18): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(19): ReLU(inplace=True)
(20): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(21): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(22): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(23): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(24): ReLU(inplace=True)
(25): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(26): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(27): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(28): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(29): ReLU(inplace=True)
(30): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(31): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(32): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(33): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(34): ReLU(inplace=True)
(35): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(36): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(37): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
(38): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(39): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(40): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(41): ReLU(inplace=True)
(42): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(43): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(44): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(45): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(46): ReLU(inplace=True)
(47): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(48): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
(0): AdaptiveAvgPool2d(output_size=1)
(1): AdaptiveMaxPool2d(output_size=1)
(2): Flatten()
(3): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(4): Dropout(p=0.5, inplace=False)
(5): Linear(in_features=1024, out_features=512, bias=True)
(6): ReLU(inplace=True)
(7): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(8): Dropout(p=0.5, inplace=False)
(9): Linear(in_features=512, out_features=16, bias=True)
(10): ReLU(inplace=True)
(11): Linear(in_features=16, out_features=1, bias=True)
)], add_time=True, silent=False, cb_fns_registered=True)
learn.show_results()
img = open_image('../input/picture4/PM_Modi_2015.jpg')
img
x = learn.predict(img)
x
(FloatItem 61.401455, tensor(61.4015), tensor(61.4015))
data_wiki_big = (data_wiki_small_src.transform(tfms, size=256)
.databunch(num_workers=0).normalize(imagenet_stats))
learn.data = data_wiki_big
data_wiki_big.train_ds[0][0].shape
torch.Size([3, 256, 256])
learn.freeze_to(-1)
learn.lr_find()
learn.recorder.plot(suggestion = True)
LR Finder is complete, type {learner_name}.recorder.plot() to see the graph.
Min numerical gradient: 9.12E-07
Min loss divided by 10: 5.75E-03
lr = 5e-4
learn.fit_one_cycle(5, max_lr=slice(lr), wd=(1e-6, 1e-4, 1e-2, 1e-1), pct_start=0.5, callbacks=[SaveModelCallback(learn)])
| epoch | train_loss | valid_loss | root_mean_squared_error | time |
|---|---|---|---|---|
| 0 | 7.837380 | 6.402583 | 9.601757 | 14:34 |
| 1 | 7.332215 | 6.078465 | 9.117067 | 14:26 |
| 2 | 7.223328 | 5.939247 | 8.911067 | 14:18 |
| 3 | 6.803191 | 5.813476 | 8.721518 | 14:18 |
| 4 | 7.061768 | 5.878111 | 8.838454 | 14:21 |
Better model found at epoch 0 with valid_loss value: 6.402583122253418. Better model found at epoch 1 with valid_loss value: 6.078464984893799. Better model found at epoch 2 with valid_loss value: 5.939246654510498. Better model found at epoch 3 with valid_loss value: 5.813475608825684.
learn.recorder.plot_lr()
learn.recorder.plot_losses()
learn.save('second_head_resnet34')
learn.load('second_head_resnet34')
Learner(data=ImageDataBunch;
Train: LabelList (34909 items)
x: ImageList
Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256)
y: FloatList
52.0,34.0,36.0,23.0,78.0
Path: ../working/wiki_utk;
Valid: LabelList (8727 items)
x: ImageList
Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256)
y: FloatList
29.0,58.0,22.0,12.0,32.0
Path: ../working/wiki_utk;
Test: None, model=AgeModel(
(agemodel): Sequential(
(0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
(1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
(4): Sequential(
(0): BasicBlock(
(conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(1): BasicBlock(
(conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(2): BasicBlock(
(conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(5): Sequential(
(0): BasicBlock(
(conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(downsample): Sequential(
(0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(1): BasicBlock(
(conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(2): BasicBlock(
(conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(3): BasicBlock(
(conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(6): Sequential(
(0): BasicBlock(
(conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(downsample): Sequential(
(0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(1): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(2): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(3): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(4): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(5): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(7): Sequential(
(0): BasicBlock(
(conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(downsample): Sequential(
(0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(1): BasicBlock(
(conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(2): BasicBlock(
(conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(8): AdaptiveConcatPool2d(
(ap): AdaptiveAvgPool2d(output_size=1)
(mp): AdaptiveMaxPool2d(output_size=1)
)
(9): Flatten()
(10): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(11): Dropout(p=0.5, inplace=False)
(12): Linear(in_features=1024, out_features=512, bias=True)
(13): ReLU(inplace=True)
(14): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(15): Dropout(p=0.5, inplace=False)
(16): Linear(in_features=512, out_features=16, bias=True)
(17): ReLU(inplace=True)
(18): Linear(in_features=16, out_features=1, bias=True)
)
), opt_func=functools.partial(<class 'torch.optim.adam.Adam'>, betas=(0.9, 0.99), eps=1e-05), loss_func=L1LossFlat(), metrics=[<function root_mean_squared_error at 0x7fcf62a9bf28>], true_wd=True, bn_wd=False, wd=0.01, train_bn=True, path=PosixPath('../working/wiki_utk'), model_dir='/temp/model/', callback_fns=[functools.partial(<class 'fastai.basic_train.Recorder'>, add_time=True, silent=False), <class 'fastai.train.ShowGraph'>, functools.partial(<class 'fastai.callbacks.mixup.MixUpCallback'>, alpha=0.2, stack_x=False, stack_y=False)], callbacks=[], layer_groups=[Sequential(
(0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
(1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
), Sequential(
(0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(6): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(7): ReLU(inplace=True)
(8): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(9): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(10): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(11): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(12): ReLU(inplace=True)
(13): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(14): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(15): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(16): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(17): ReLU(inplace=True)
(18): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(19): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(20): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
(21): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(22): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(23): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(24): ReLU(inplace=True)
(25): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(26): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(27): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(28): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(29): ReLU(inplace=True)
(30): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(31): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(32): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(33): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(34): ReLU(inplace=True)
(35): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(36): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
(0): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
(6): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(7): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(8): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(9): ReLU(inplace=True)
(10): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(11): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(13): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(14): ReLU(inplace=True)
(15): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(16): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(17): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(18): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(19): ReLU(inplace=True)
(20): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(21): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(22): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(23): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(24): ReLU(inplace=True)
(25): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(26): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(27): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(28): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(29): ReLU(inplace=True)
(30): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(31): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(32): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(33): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(34): ReLU(inplace=True)
(35): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(36): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(37): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
(38): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(39): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(40): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(41): ReLU(inplace=True)
(42): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(43): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(44): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(45): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(46): ReLU(inplace=True)
(47): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(48): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
(0): AdaptiveAvgPool2d(output_size=1)
(1): AdaptiveMaxPool2d(output_size=1)
(2): Flatten()
(3): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(4): Dropout(p=0.5, inplace=False)
(5): Linear(in_features=1024, out_features=512, bias=True)
(6): ReLU(inplace=True)
(7): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(8): Dropout(p=0.5, inplace=False)
(9): Linear(in_features=512, out_features=16, bias=True)
(10): ReLU(inplace=True)
(11): Linear(in_features=16, out_features=1, bias=True)
)], add_time=True, silent=False, cb_fns_registered=True)
img = open_image('../input/picture4/PM_Modi_2015.jpg')
img
learn.predict(img)
(FloatItem 61.38315, tensor(61.3831), tensor(61.3831))
x,y = next(iter(learn.data.train_dl))
x.shape
torch.Size([64, 3, 256, 256])
learn.unfreeze()
learn.lr_find()
learn.recorder.plot(suggestion = True)
LR Finder is complete, type {learner_name}.recorder.plot() to see the graph.
Min numerical gradient: 9.12E-07
Min loss divided by 10: 3.02E-06
learn.fit_one_cycle(5, max_lr=slice(1e-6,1e-4), wd=(1e-6, 1e-4, 1e-2, 1e-1), callbacks=[SaveModelCallback(learn)], pct_start=0.5)
| epoch | train_loss | valid_loss | root_mean_squared_error | time |
|---|---|---|---|---|
| 0 | 6.700283 | 5.710420 | 8.648839 | 14:52 |
| 1 | 6.323674 | 5.421929 | 8.128598 | 15:00 |
| 2 | 6.052965 | 5.245904 | 7.984351 | 14:55 |
| 3 | 5.895456 | 5.199911 | 7.924792 | 14:54 |
| 4 | 5.728145 | 5.103240 | 7.743491 | 14:55 |
Better model found at epoch 0 with valid_loss value: 5.71042013168335. Better model found at epoch 1 with valid_loss value: 5.421929359436035. Better model found at epoch 2 with valid_loss value: 5.245903968811035. Better model found at epoch 3 with valid_loss value: 5.199910640716553. Better model found at epoch 4 with valid_loss value: 5.103240489959717.
learn.save('second_body_resnet34')
learn.load('second_body_resnet34')
Learner(data=ImageDataBunch;
Train: LabelList (34909 items)
x: ImageList
Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256)
y: FloatList
52.0,34.0,36.0,23.0,78.0
Path: ../working/wiki_utk;
Valid: LabelList (8727 items)
x: ImageList
Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256)
y: FloatList
29.0,58.0,22.0,12.0,32.0
Path: ../working/wiki_utk;
Test: None, model=AgeModel(
(agemodel): Sequential(
(0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
(1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
(4): Sequential(
(0): BasicBlock(
(conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(1): BasicBlock(
(conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(2): BasicBlock(
(conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(5): Sequential(
(0): BasicBlock(
(conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(downsample): Sequential(
(0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(1): BasicBlock(
(conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(2): BasicBlock(
(conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(3): BasicBlock(
(conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(6): Sequential(
(0): BasicBlock(
(conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(downsample): Sequential(
(0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(1): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(2): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(3): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(4): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(5): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(7): Sequential(
(0): BasicBlock(
(conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(downsample): Sequential(
(0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(1): BasicBlock(
(conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(2): BasicBlock(
(conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(8): AdaptiveConcatPool2d(
(ap): AdaptiveAvgPool2d(output_size=1)
(mp): AdaptiveMaxPool2d(output_size=1)
)
(9): Flatten()
(10): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(11): Dropout(p=0.5, inplace=False)
(12): Linear(in_features=1024, out_features=512, bias=True)
(13): ReLU(inplace=True)
(14): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(15): Dropout(p=0.5, inplace=False)
(16): Linear(in_features=512, out_features=16, bias=True)
(17): ReLU(inplace=True)
(18): Linear(in_features=16, out_features=1, bias=True)
)
), opt_func=functools.partial(<class 'torch.optim.adam.Adam'>, betas=(0.9, 0.99), eps=1e-05), loss_func=L1LossFlat(), metrics=[<function root_mean_squared_error at 0x7fcf62a9bf28>], true_wd=True, bn_wd=False, wd=0.01, train_bn=True, path=PosixPath('../working/wiki_utk'), model_dir='/temp/model/', callback_fns=[functools.partial(<class 'fastai.basic_train.Recorder'>, add_time=True, silent=False), <class 'fastai.train.ShowGraph'>, functools.partial(<class 'fastai.callbacks.mixup.MixUpCallback'>, alpha=0.2, stack_x=False, stack_y=False)], callbacks=[], layer_groups=[Sequential(
(0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
(1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
), Sequential(
(0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(6): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(7): ReLU(inplace=True)
(8): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(9): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(10): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(11): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(12): ReLU(inplace=True)
(13): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(14): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(15): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(16): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(17): ReLU(inplace=True)
(18): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(19): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(20): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
(21): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(22): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(23): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(24): ReLU(inplace=True)
(25): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(26): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(27): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(28): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(29): ReLU(inplace=True)
(30): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(31): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(32): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(33): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(34): ReLU(inplace=True)
(35): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(36): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
(0): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
(6): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(7): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(8): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(9): ReLU(inplace=True)
(10): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(11): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(13): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(14): ReLU(inplace=True)
(15): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(16): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(17): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(18): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(19): ReLU(inplace=True)
(20): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(21): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(22): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(23): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(24): ReLU(inplace=True)
(25): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(26): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(27): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(28): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(29): ReLU(inplace=True)
(30): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(31): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(32): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(33): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(34): ReLU(inplace=True)
(35): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(36): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(37): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
(38): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(39): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(40): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(41): ReLU(inplace=True)
(42): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(43): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(44): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(45): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(46): ReLU(inplace=True)
(47): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(48): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
(0): AdaptiveAvgPool2d(output_size=1)
(1): AdaptiveMaxPool2d(output_size=1)
(2): Flatten()
(3): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(4): Dropout(p=0.5, inplace=False)
(5): Linear(in_features=1024, out_features=512, bias=True)
(6): ReLU(inplace=True)
(7): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(8): Dropout(p=0.5, inplace=False)
(9): Linear(in_features=512, out_features=16, bias=True)
(10): ReLU(inplace=True)
(11): Linear(in_features=16, out_features=1, bias=True)
)], add_time=True, silent=False, cb_fns_registered=True)
learn.show_results()
img = open_image('../input/picture4/PM_Modi_2015.jpg')
img
x = learn.predict(img)
x
(FloatItem 63.15648, tensor(63.1565), tensor(63.1565))
learn.save('third_body_resnet34')
learn.load('third_body_resnet34')
Learner(data=ImageDataBunch;
Train: LabelList (34909 items)
x: ImageList
Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256)
y: FloatList
52.0,34.0,36.0,23.0,78.0
Path: ../working/wiki_utk;
Valid: LabelList (8727 items)
x: ImageList
Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256)
y: FloatList
29.0,58.0,22.0,12.0,32.0
Path: ../working/wiki_utk;
Test: None, model=AgeModel(
(agemodel): Sequential(
(0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
(1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
(4): Sequential(
(0): BasicBlock(
(conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(1): BasicBlock(
(conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(2): BasicBlock(
(conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(5): Sequential(
(0): BasicBlock(
(conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(downsample): Sequential(
(0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(1): BasicBlock(
(conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(2): BasicBlock(
(conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(3): BasicBlock(
(conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(6): Sequential(
(0): BasicBlock(
(conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(downsample): Sequential(
(0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(1): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(2): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(3): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(4): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(5): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(7): Sequential(
(0): BasicBlock(
(conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(downsample): Sequential(
(0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(1): BasicBlock(
(conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(2): BasicBlock(
(conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(8): AdaptiveConcatPool2d(
(ap): AdaptiveAvgPool2d(output_size=1)
(mp): AdaptiveMaxPool2d(output_size=1)
)
(9): Flatten()
(10): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(11): Dropout(p=0.5, inplace=False)
(12): Linear(in_features=1024, out_features=512, bias=True)
(13): ReLU(inplace=True)
(14): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(15): Dropout(p=0.5, inplace=False)
(16): Linear(in_features=512, out_features=16, bias=True)
(17): ReLU(inplace=True)
(18): Linear(in_features=16, out_features=1, bias=True)
)
), opt_func=functools.partial(<class 'torch.optim.adam.Adam'>, betas=(0.9, 0.99), eps=1e-05), loss_func=L1LossFlat(), metrics=[<function root_mean_squared_error at 0x7fcf62a9bf28>], true_wd=True, bn_wd=False, wd=0.01, train_bn=True, path=PosixPath('../working/wiki_utk'), model_dir='/temp/model/', callback_fns=[functools.partial(<class 'fastai.basic_train.Recorder'>, add_time=True, silent=False), <class 'fastai.train.ShowGraph'>, functools.partial(<class 'fastai.callbacks.mixup.MixUpCallback'>, alpha=0.2, stack_x=False, stack_y=False)], callbacks=[], layer_groups=[Sequential(
(0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
(1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
), Sequential(
(0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(6): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(7): ReLU(inplace=True)
(8): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(9): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(10): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(11): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(12): ReLU(inplace=True)
(13): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(14): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(15): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(16): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(17): ReLU(inplace=True)
(18): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(19): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(20): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
(21): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(22): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(23): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(24): ReLU(inplace=True)
(25): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(26): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(27): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(28): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(29): ReLU(inplace=True)
(30): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(31): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(32): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(33): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(34): ReLU(inplace=True)
(35): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(36): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
(0): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
(6): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(7): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(8): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(9): ReLU(inplace=True)
(10): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(11): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(12): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(13): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(14): ReLU(inplace=True)
(15): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(16): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(17): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(18): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(19): ReLU(inplace=True)
(20): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(21): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(22): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(23): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(24): ReLU(inplace=True)
(25): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(26): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(27): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(28): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(29): ReLU(inplace=True)
(30): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(31): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(32): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(33): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(34): ReLU(inplace=True)
(35): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(36): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(37): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
(38): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(39): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(40): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(41): ReLU(inplace=True)
(42): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(43): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(44): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(45): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(46): ReLU(inplace=True)
(47): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(48): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
), Sequential(
(0): AdaptiveAvgPool2d(output_size=1)
(1): AdaptiveMaxPool2d(output_size=1)
(2): Flatten()
(3): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(4): Dropout(p=0.5, inplace=False)
(5): Linear(in_features=1024, out_features=512, bias=True)
(6): ReLU(inplace=True)
(7): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(8): Dropout(p=0.5, inplace=False)
(9): Linear(in_features=512, out_features=16, bias=True)
(10): ReLU(inplace=True)
(11): Linear(in_features=16, out_features=1, bias=True)
)], add_time=True, silent=False, cb_fns_registered=True)
class SaveFeatures():
features=None
def __init__(self, m):
self.hook = m.register_forward_hook(self.hook_fn)
self.features = None
def hook_fn(self, module, input, output):
out = output.detach().cpu().numpy()
if isinstance(self.features, type(None)):
self.features = out
else:
self.features = np.row_stack((self.features, out))
def remove(self):
self.hook.remove()
learn.model
AgeModel(
(agemodel): Sequential(
(0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
(1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
(4): Sequential(
(0): BasicBlock(
(conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(1): BasicBlock(
(conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(2): BasicBlock(
(conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(5): Sequential(
(0): BasicBlock(
(conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(downsample): Sequential(
(0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(1): BasicBlock(
(conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(2): BasicBlock(
(conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(3): BasicBlock(
(conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(6): Sequential(
(0): BasicBlock(
(conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(downsample): Sequential(
(0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(1): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(2): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(3): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(4): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(5): BasicBlock(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(7): Sequential(
(0): BasicBlock(
(conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(downsample): Sequential(
(0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(1): BasicBlock(
(conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(2): BasicBlock(
(conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU(inplace=True)
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(8): AdaptiveConcatPool2d(
(ap): AdaptiveAvgPool2d(output_size=1)
(mp): AdaptiveMaxPool2d(output_size=1)
)
(9): Flatten()
(10): BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(11): Dropout(p=0.5, inplace=False)
(12): Linear(in_features=1024, out_features=512, bias=True)
(13): ReLU(inplace=True)
(14): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(15): Dropout(p=0.5, inplace=False)
(16): Linear(in_features=512, out_features=16, bias=True)
(17): ReLU(inplace=True)
(18): Linear(in_features=16, out_features=1, bias=True)
)
)
# Second last layer of the model
learn.model.agemodel[16]
Linear(in_features=512, out_features=16, bias=True)
len(data_wiki_big.train_ds.items), len(data_wiki_big.valid_ds.items)
(34909, 8727)
sf = SaveFeatures(learn.model.agemodel[16])
_= learn.get_preds(data_wiki_big.train_ds)
_= learn.get_preds(DatasetType.Valid)
len(sf.features)
43636
img_path = [str(x) for x in (list(data_wiki_big.train_ds.items) +list(data_wiki_big.valid_ds.items))]
label = [x for x in (list(data_wiki_big.train_ds.y.items) +list(data_wiki_big.valid_ds.y.items))]
label_id = [x for x in (list(data_wiki_big.train_ds.y.items) +list(data_wiki_big.valid_ds.y.items))]
df_new = pd.DataFrame({'img_path': img_path, 'label': label, 'label_id': label_id})
df_new
| img_path | label | label_id | |
|---|---|---|---|
| 0 | ../working/wiki_utk/./10002702_1960-11-09_2012... | 52.0 | 52.0 |
| 1 | ../working/wiki_utk/./10003541_1937-09-27_1971... | 34.0 | 34.0 |
| 2 | ../working/wiki_utk/./1000456_1933-06-12_1969.jpg | 36.0 | 36.0 |
| 3 | ../working/wiki_utk/./10004882_1987-05-16_2010... | 23.0 | 23.0 |
| 4 | ../working/wiki_utk/./1000781_1930-07-07_2008.jpg | 78.0 | 78.0 |
| ... | ... | ... | ... |
| 43631 | ../working/wiki_utk/./007148.jpg | 20.0 | 20.0 |
| 43632 | ../working/wiki_utk/./4948808_1983-11-16_2007.jpg | 24.0 | 24.0 |
| 43633 | ../working/wiki_utk/./6_1_4_20170119204208503.... | 6.0 | 6.0 |
| 43634 | ../working/wiki_utk/./26_0_4_20170117154446109... | 26.0 | 26.0 |
| 43635 | ../working/wiki_utk/./18_0_4_20170103201305015... | 18.0 | 18.0 |
43636 rows × 3 columns
array = np.array(sf.features)
x=array.tolist()
df_new['img_repr'] = x
df_new
| img_path | label | label_id | img_repr | |
|---|---|---|---|---|
| 0 | ../working/wiki_utk/./10002702_1960-11-09_2012... | 52.0 | 52.0 | [72.47952270507812, -8.392868041992188, -9.617... |
| 1 | ../working/wiki_utk/./10003541_1937-09-27_1971... | 34.0 | 34.0 | [72.01614379882812, -7.710638999938965, -8.909... |
| 2 | ../working/wiki_utk/./1000456_1933-06-12_1969.jpg | 36.0 | 36.0 | [69.15504455566406, -7.593712329864502, -8.700... |
| 3 | ../working/wiki_utk/./10004882_1987-05-16_2010... | 23.0 | 23.0 | [49.39116668701172, -7.189101696014404, -7.045... |
| 4 | ../working/wiki_utk/./1000781_1930-07-07_2008.jpg | 78.0 | 78.0 | [134.6729736328125, -16.601808547973633, -19.6... |
| ... | ... | ... | ... | ... |
| 43631 | ../working/wiki_utk/./007148.jpg | 20.0 | 20.0 | [42.290016174316406, -7.002415657043457, -6.35... |
| 43632 | ../working/wiki_utk/./4948808_1983-11-16_2007.jpg | 24.0 | 24.0 | [28.6945743560791, -7.378058910369873, -5.9204... |
| 43633 | ../working/wiki_utk/./6_1_4_20170119204208503.... | 6.0 | 6.0 | [49.36155700683594, -6.653528213500977, -6.824... |
| 43634 | ../working/wiki_utk/./26_0_4_20170117154446109... | 26.0 | 26.0 | [49.14276885986328, -5.3559465408325195, -5.08... |
| 43635 | ../working/wiki_utk/./18_0_4_20170103201305015... | 18.0 | 18.0 | [21.474647521972656, -6.6602959632873535, -4.7... |
43636 rows × 4 columns
df_new.shape
(43636, 4)
from annoy import AnnoyIndex
f = len(df_new['img_repr'][0])
t = AnnoyIndex(f, metric='euclidean')
ntree = 1000
for i, vector in enumerate(df_new['img_repr']):
t.add_item(i, vector)
_ = t.build(ntree)
import time
def get_similar_images_annoy(img_index, num):
start = time.time()
base_img_id, base_vector, base_label = df_new.iloc[img_index, [0,3,1]]
similar_img_ids = t.get_nns_by_item(img_index, num)
end = time.time()
print(f'{(end - start) * 1000} ms')
return base_img_id, base_label, df_new.iloc[similar_img_ids]
num = 89
base_image, base_label, similar_images_df = get_similar_images_annoy(890, num)
70.15395164489746 ms
print(base_label)
open_image(base_image)
21.0
similar_images_df
| img_path | label | label_id | img_repr | |
|---|---|---|---|---|
| 890 | ../working/wiki_utk/./950234_1941-03-06_1962.jpg | 21.0 | 21.0 | [44.051395416259766, -8.610692024230957, -7.61... |
| 10285 | ../working/wiki_utk/./1575279_1978-12-28_2014.jpg | 36.0 | 36.0 | [44.23878479003906, -8.540730476379395, -7.615... |
| 40921 | ../working/wiki_utk/./23465846_1989-07-10_2010... | 21.0 | 21.0 | [44.385948181152344, -8.653063774108887, -7.68... |
| 11172 | ../working/wiki_utk/./4182570_1978-06-26_2012.jpg | 34.0 | 34.0 | [44.501441955566406, -8.626776695251465, -7.64... |
| 34427 | ../working/wiki_utk/./007004.jpg | 27.0 | 27.0 | [44.107200622558594, -8.570219993591309, -7.66... |
| ... | ... | ... | ... | ... |
| 3505 | ../working/wiki_utk/./25239834_1992-01-23_2015... | 23.0 | 23.0 | [44.192176818847656, -8.403630256652832, -7.45... |
| 40902 | ../working/wiki_utk/./1691972_1933-02-18_1968.jpg | 35.0 | 35.0 | [45.43599319458008, -8.536643981933594, -7.595... |
| 12401 | ../working/wiki_utk/./1190101_1979-11-13_2010.jpg | 31.0 | 31.0 | [45.04903793334961, -8.50577449798584, -7.5224... |
| 40885 | ../working/wiki_utk/./4094157_1990-03-01_2009.jpg | 19.0 | 19.0 | [43.83957290649414, -8.407843589782715, -7.513... |
| 13491 | ../working/wiki_utk/./589278_1897-01-23_1942.jpg | 45.0 | 45.0 | [45.358192443847656, -8.5722074508667, -7.6586... |
89 rows × 4 columns
data_src = (ImageList.from_df(similar_images_df, path=".",cols=['img_path'], folder='.' )
.split_none()
.label_from_df(cols=['label'],label_cls=FloatList))
data = (data_src.transform(tfms, resize_method=ResizeMethod.CROP, padding_mode='border', size=256)
.databunch(bs=num,num_workers=0).normalize(imagenet_stats))
data.show_batch(2)